YSK: If you're going to use AI for knowledge work, make it disagree with you

10 days ago by Artisian to c/youshouldknow

Prompting AI models to contradict your ideas can improve your thinking about a problem.

We're seeing a lot of students, teachers, and scientists produce more, and worse, outputs because of improper AI use. It's possible to use it better: Have it imitate a critic. Learning and creativity come from getting criticism and experiencing friction.

We found that when knowledge workers intentionally use AI to challenge their ideas – to generate friction – they can significantly improve their performance. Previous research on human-AI collaboration for loan evaluations had similar results.

A study I was not involved with found that creative writers produce better copy when they use AI as a sounding board rather than to ghostwrite.

besselj 55 points 10 days ago

Or don't use AI for knowledge work when being correct is important.

path: 0 26131021, hotness: undefined, score: 55, children: 11
Artisian 6 points 10 days ago

Sometimes knowledge work is about learning. That's also a nice time to skip AI. But I can see from the data that student's aren't doing that, so we harm-reduce.

path: 0 26131021 26131114, hotness: undefined, score: 6, children: 10
naught101 8 points 10 days ago

Using it for learning stuff that's already well known (e.g. beginner level coding) is fine.

Using it for "knowledge work" is a bad idea idea, because AI pushes toward the mean, while research by definition is focused on new edges of knowledge. This is is exactly the space where AI will introduce biases and untruths that will be very hard to spot. Using it also reduces critical thinking ability which is critical in this space.

path: 0 26131021 26131114 26134977, hotness: undefined, score: 8, children: 9
Artisian 2 points 10 days ago

If you're doing knowledge work, you are an expert. You're very likely to catch biases and untruths (and those you don't catch, humanity couldn't catch. These concerns are just as applicable to a human by themselves.)

path: 0 26131021 26131114 26134977 26135240, hotness: undefined, score: 2, children: 8
naught101 11 points 9 days ago

Sorry, but that's just not true. Researchers might be experts in teir field generally, but day-to-day work is very rarely focused on well-known parts of their domain, because the whole point is to obtain new knowledge.

Source: I'm a climate scientist, have been in and around academia for nearly 20 years.

path: 0 26131021 26131114 26134977 26135240 26137251, hotness: undefined, score: 11, children: 7
HubertManne 15 points 10 days ago

Someone else I know said this in a conference and I disagree. You should instruct it to not do any extraneous language and give back data in the most impartial way possible. You should use it for things you know well so that you can see how it fails and double check any results you are going to use. Having it purpsefully be adversarial is not going to improve what you get out of it and will just waste time.

path: 0 26131388, hotness: undefined, score: 15, children: 6
Artisian 1 point 10 days ago

Thoughts on the study from the article? link

path: 0 26131388 26134689, hotness: undefined, score: 1, children: 4
HubertManne -9 points 10 days ago

oh I won't go to links off the fediverse.

path: 0 26131388 26134689 26135263, hotness: undefined, score: -9, children: 3
Artisian 3 points 10 days ago

lol (This is why we can't have nice things)

path: 0 26131388 26134689 26135263 26135284, hotness: undefined, score: 3, children: 2
HubertManne -3 points 10 days ago

yeah. I mean copy paste is so easy. 100% agree.

path: 0 26131388 26134689 26135263 26135284 26135312, hotness: undefined, score: -3, children: 1
hendrik 1 point 10 days ago

Dunno, does that help other people? Every time I use it for programming and don't specifically tell it what to do, it'll just come up with assumptions. Might go the right direction. Or some random other direction and miss what'd be important factors.
Telling it for example to find issues with the following algorithm / implementation (and then copy-pasting my initial idea) has indeed helped me to find edge cases and stuff like that.

I also think there's an issue with XY-Problems. Either AI suggested X to me. Or it was my fault because I asked for X. But I've ended up in several wild goose chases before, and it's rare AI stops and points out I'm on the wrong path because I need Y.

Extraneous drivel is very annoying, though. Seems to me especially ChatGPT likes to spam multiple solutions to any question and you can often disregard the first two pages of "ideas" it came up how to NOT do it.

path: 0 26131388 26132251, hotness: undefined, score: 1, children: 0
CapuccinoCoretto 15 points 10 days ago

Or better yet, don't! Think for yourself while you still can.

path: 0 26131065, hotness: undefined, score: 15, children: 7
weps -3 points 9 days ago

yeah but you're very biased about what you think

it actually is really refreshing to get a LLM to break down your arguments and ideas

cognitive friction really gets the best out of your brain even if the AI is only averagely good at disproving your ideas

thinking for yourself and using AI isn't mutually exclusive

path: 0 26131065 26137950, hotness: undefined, score: -3, children: 6
FearMeAndDecay 5 points 9 days ago

This is what peer review, teacher feedback, and group workshops are for. In college I didn't need an llm to critique my work. That was the job of my classmates, and mine for their writing. And for those outside of college, you can still just seek out writing circles, either irl or online. It takes some work, but you'll get better feedback from real humans who can actually feel the emotion in your writing. You can also seek out similar communities for other hobbies. The only reason for someone to pick an ai critique over a human one is convenience and the cost is way too high to justify that

path: 0 26131065 26137950 26143283, hotness: undefined, score: 5, children: 3
weps 1 point 9 days ago

I'm sure the subjects of the research mentioned in the article do all that.

I can say AI is being used for a lot of stupid wasteful things. I don't know what it costs for AI to seemingly improve our knowledge workers, but to me that is worth something.

path: 0 26131065 26137950 26143283 26143637, hotness: undefined, score: 1, children: 1
Artisian -1 points 8 days ago

Knowledge work is generally pretty expensive; it's done in dense places, uses a lot of air travel, has pretty big long-term consequences. Even the high-end estimates on AI costs are probably worth modest gains for knowledge workers.

path: 0 26131065 26137950 26143283 26143637 26153004, hotness: undefined, score: -1, children: 0
Artisian -2 points 8 days ago

Idk about you, but my mentors and peers are all busy stressing over the polycrisis.

path: 0 26131065 26137950 26143283 26152990, hotness: undefined, score: -2, children: 0
Swedneck 4 points 9 days ago

mmm yes, using the thing famous for being fundamentally programmed to be a sycophant to challenge your ideas..

Did you have the AI challenge that idea?

path: 0 26131065 26137950 26139707, hotness: undefined, score: 4, children: 1
weps 1 point 9 days ago

don't antromorphize the computer sycopathy is just how they programmed the thing all the information to heavily challenge your ideas is in there

try it

tell it you think using an AI to challenge your ideas is great (the opposite of what you think)

path: 0 26131065 26137950 26139707 26140030, hotness: undefined, score: 1, children: 0
naught101 9 points 10 days ago

AI is going to tank academia. It's already going down hill after decades of corporate-style management.

path: 0 26134851, hotness: undefined, score: 9, children: 0
MountainVeil 5 points 9 days ago

I've done this. Hilariously frustrating when it starts doubling down on hallucinations.

path: 0 26143593, hotness: undefined, score: 5, children: 1
Artisian -3 points 8 days ago

As I've been messing around with it, I am becoming a fan of 'hallucination -> new context'.

path: 0 26143593 26153028, hotness: undefined, score: -3, children: 0
weps 4 points 9 days ago

prompt like this:

"what did Nietzsche think was the influence on western society of the victory of jewish christian moral on the ancient greek values

and why was he so stupid about it"

path: 0 26137864, hotness: undefined, score: 4, children: 0
Rhynoplaz 4 points 10 days ago

Yeah, right. I've tried to do this everywhere I've worked.

Why do we think managers are suddenly going to appreciate being told that they're wrong just because a robot does it?

path: 0 26131360, hotness: undefined, score: 4, children: 0
JTode 3 points 10 days ago

I'm not walking around touting it, cause this is a time of battling cults. But I have been using an AI as a tutor to teach me how to write C code, with an eye towards kernel contributions and other things that require that speed. Obviously it's a more marketable dev skill, even in this time, than the Python I know. It is very good at teaching, actually. Too bad about us all perishing by asphyxiation.

path: 0 26134496, hotness: undefined, score: 3, children: 4
kungen 8 points 9 days ago

I wouldn't trust a professor who lies to me as often as LLMs do.

path: 0 26134496 26138013, hotness: undefined, score: 8, children: 3
Swedneck 3 points 9 days ago

it's not even lies it's just not being able to tell what is true because you're not actually conscious

path: 0 26134496 26138013 26139691, hotness: undefined, score: 3, children: 1
Artisian -2 points 8 days ago

(Why is consciousness required to tell true/false statements? I think I'd even say 'that sign is dishonest/wrong/a liar', regardless of the fact that the sign is very much dead.)

path: 0 26134496 26138013 26139691 26153022, hotness: undefined, score: -2, children: 0
JTode 2 points 9 days ago

Programming is a very non-fuzzy topic; it's no surprise that as the great delusion shakes out, generated art is too despised to be a significant revenue stream, and so far as I can tell the generated human text is still amateurishly bad.

But it's a fact that these models have been trained on just about every bit of publicly-available code, which by and large, is good, working code. Not every bit of it, but enough that I have yet to see the thing spit out bad syntax... though it catches my bad syntax every time.

I have not and will never try to use it to write code for me, but it's a smashing tutor, and I could certainly see an experienced dev who knows exactly what they want, handing off the building of tedious functions and stuff, piece by piece, and getting back good, compact code. I believe that the problems we're having are to do with someone handing off the building of an entire website, or an entire app, that's where you end up with runaway dependencies, bloated and badly-written functions, etc.

Now, it still goes idiot all the time. I point out that my push function is failing because the ringbuffer_is_full check is returning true every time (I'm very dumb when I'm first starting out on anything), it "sees" the problem and tells me correctly what to do to fix it, but it talks about the function returning false every time. Correct answer, but the dressing on the salad is wrong. This only happens, though, after I've been in the same chat for a while (context window overflow).

In other words, it's a tool that humans can use, while keeping an eye for that kinda thing, but it certainly will not replace anyone, though we're going to absorb a lot of damage from stupid rich people trying.

In other news, I had a dream last night where I was hanging out with an old ex, and when she showed up, she (somehow) told me that not only had she gone blind since we last talked, she had then gotten a brain tumour and had her head amputated. So I was walking around with my headless ex all day in this dream. I don't dislike this ex, we were just walking around (how?) and talking to people (HOW?).

I wonder if this is a product of me conversing with something that has no consciousness so much this last week. Or maybe it's the Ed Gein series we're watching, I dunno.

path: 0 26134496 26138013 26142651, hotness: undefined, score: 2, children: 0
JohnnyEnzyme 3 points 10 days ago

I've actually been able to shape GPT this way via a bunch of global settings I've invoked over the years. Don't get me wrong-- it's a PITA, and pretty much a never-ending process, but I think I have two dozen settings now it follows as best it can in order to provide something like a 'pros and con' list for any opinion it spits out. This cuts way down on its classic tendency to hallucinate bullshit and spit out half-baked / incorrect replies.

(ready for the downvotes now assuming that I love AI as a general concept πŸ˜…)

path: 0 26133444, hotness: undefined, score: 3, children: 0
Pratai 3 points 7 days ago

Or don’t use AI for any work that matters.

path: 0 26167577, hotness: undefined, score: 3, children: 0
HeHoXa 0 points 8 days ago

Don't use it for conclusions at all.

Use it to expand your vocabulary on the topic and as a search engine supplement to find sources

path: 0 26160551, hotness: undefined, score: 0, children: 0
youshouldknow
youshouldknow

@lemmy.world

login for more options
47865
779
4106

YSK - for all the things that can make your life easier!

The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:

Rules (interactive)


Rule 1- All posts must begin with YSK.

All posts must begin with YSK. If you're a Mastodon user, then include YSK after @youshouldknow. This is a community to share tips and tricks that will help you improve your life.



Rule 2- Your post body text must include the reason **WHY** YSK:

In your post's text body, you must include the reason "Why" YSK: It’s helpful for readability, and informs readers about the importance of the content.



Rule 3- Do not seek mental, medical and professional help here.

Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.



Rule 4- No self promotion or upvote-farming of any kind.

That's it.



Rule 5- No baiting or sealioning or promoting an agenda.

Posts and comments which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.



Rule 6- Regarding non-YSK posts.

Provided it is about the community itself, you may post non-YSK posts using the [META] tag on your post title.



Rule 7- You can't harass or disturb other members.

If you harass or discriminate against any individual member, you will be removed.

If you are a member, sympathizer or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people and you were provably vocal about your hate, then you will be banned on sight.

For further explanation, clarification and feedback about this rule, you may follow this link.



Rule 8- All comments should try to stay relevant to their parent content.

Rule 9- Reposts from other platforms are not allowed.

Let everyone have their own content.



Rule 10- The majority of bots aren't allowed to participate here.

Unless included in our Whitelist for Bots, your bot will not be allowed to participate in this community. To have your bot whitelisted, please contact the moderators for a short review.



Rule 11- Posts must actually be true:

Disinformation, trolling, and being misleading will not be tolerated. Repeated or egregious attempts will earn you a ban. This also applies to filing reports: If you continually file false reports YOU WILL BE BANNED! We can see who reports what, and shenanigans will not be tolerated. We are not here to ban people who said something you don't like.



Rule 12- Linking directly to a video as a source is not acceptable.

Video only posts are responsible for an outsized portion of rules violations. Your post must contain text and meet the requirements of the previous rules.



Rule 13- If you file a report, include what specific rule is being violated and how.

Partnered Communities:

You can view our partnered communities list by following this link. To partner with our community and be included, you are free to message the moderators or comment on a pinned post.

Community Moderation

For inquiry on becoming a moderator of this community, you may comment on the pinned post of the time, or simply shoot a message to the current moderators.

Credits

Our icon(masterpiece) was made by @clen15!

go to feed...