OpenAI says it is investigating reports ChatGPT has become ‘lazy’

3 years ago by L4sBot to c/technology

OpenAI says it is investigating reports ChatGPT has become ‘lazy’::OpenAI says it is investigating complaints about ChatGPT having become “lazy”.

rtfm_modular 124 points 3 years ago

Yep, I spent a month refactoring a few thousand lines of code using GPT4 and I felt like I was working with the best senior developer with infinite patience and availability.

I could vaguely describe what I was after and it would identify the established programming patterns and provide examples based on all the code snippets I fed it. It was amazing and a little terrifying what an LLM is capable of. It didn’t write the code for me but it increased my productivity 2 fold... I’m a developer now a getting rusty being 5 years into management rather than delivering functional code, so just having that copilot was invaluable.

Then one day it just stopped. It lost all context for my project. I asked what it thought what we were working on and it replied with something to do with TCP relays instead of my little Lua pet project dealing with music sequencing and MIDI processing… not even close to the fucking ballpark’s overflow lot.

It’s like my trusty senior developer got smashed in the head with a brick. And as described, would just give me nonsense hand wavy answers.

path: 0 5934028, hotness: undefined, score: 124, children: 7

BleatingZombie 39 points 3 years ago

"ChatGPT Caught Faking On-Site Injury for L&I"

path: 0 5934028 5935957, hotness: undefined, score: 39, children: 0

backgroundcow 18 points 3 years ago

Was this around the time right after "custom GPTs" was introduced? I've seen posts since basically the beginning of ChatGPT claming it got stupid and thinking it was just confirmation bias. But somewhere around that point I felt a shift myself in GPT4:s ability to program; where it before found clever solutions to difficult problems, it now often struggles with basics.

path: 0 5934028 5938962, hotness: undefined, score: 18, children: 5

Linkerbaan 19 points 3 years ago

Maybe they're crippling it so when GPT5 releases it looks better. Like Apple did with cpu throttling of older iphones

path: 0 5934028 5938962 5939737, hotness: undefined, score: 19, children: 3

tagliatelle 17 points 3 years ago

They probably have to scale down the resources used for each query as they can't scale up their infrastructure to handle the load.

path: 0 5934028 5938962 5939737 5941797, hotness: undefined, score: 17, children: 2

backgroundcow 4 points 3 years ago

This is my guess as well. They have been limiting new signups for the paid service for a long time, which must mean they are overloaded; and then it makes a lot of sense to just degrade the quality of GPT-4 so they can serve all paying users. I just wish there was a way to know the "quality level" the service is operating at.

path: 0 5934028 5938962 5939737 5941797 5947510, hotness: undefined, score: 4, children: 0

monkeyslikebananas2 2 points 3 years ago

This is most likely the answer. Management saw the revenue and cost and said, “whoa! Turn all that unnecessary stuff off!”

path: 0 5934028 5938962 5939737 5941797 5943155, hotness: undefined, score: 2, children: 0

Meowoem 1 point 3 years ago

I do think part of it is expectation creep but also that it's got better at some harder elements which aren't as noticeable - it used to invent functions which should exist but don't, I haven't seen it do that in a while but it does seem to have limited the scope it can work with. I think it's probably like how with images you can have it make great images OR strictly obey the prompt but the more you want it to do one the less it can do the other.

I've been using 3.5 to help code and it's incredibly useful for things it's good at like reminding me what a certain function call does and what my options are with it, it's got much better at that and tiny scripts like 'a python script that reads all the files in a folder and sorts the big images into a separate folder' or something like that. Getting it to handle anything with more complexity it's got worse at, it was never great at it tbh so I think maybe it's getting to s block where now it knows it can't do it so rejects the answers with critical failures (like when it makes up function of a standard library because it'd be useful) and settles on a weaker but less wrong one - a lot of the making up functions errors were easy to fix because you could just say 'pil doesn't have a function to do that can you write one'

So yeah I don't think it's really getting worse but there are tradeoffs - if only openAI lived by any of the principles they claimed when setting up and naming themselves then we'd be able to experiment and explore different usage methods for different tasks just like people do with stable diffusion. But capitalists are going to lie, cheat, and try to monopolize so we're stuck guessing.

path: 0 5934028 5938962 5942343, hotness: undefined, score: 1, children: 0

paddirn 111 points 3 years ago

First it just starts making shit up, then lying about it, now it’s just at the stage where it’s like, “Fuck this shit.” It’s becoming more human by the day.

path: 0 5933702, hotness: undefined, score: 111, children: 1

MisterChief 22 points 3 years ago

Human. After all.

path: 0 5933702 5935742, hotness: undefined, score: 22, children: 0

Enkers 86 points 3 years ago

AI systems such as ChatGPT are notoriously costly for the companies that run them, and so giving detailed answers to questions can require considerable processing power and computing time.

This is the crux of the problem. Here's my speculation on OpenAI's business model:

Build good service to attract users, operate at a loss.
Slowly degrade service to stem the bleeding.
Begin introducing advertised content.
Further enshitify.

It's basically the Google playbook. Pretend to be good until people realize you're just trying to stuff ads down their throats for the sweet advertising revenue.

path: 0 5934127, hotness: undefined, score: 86, children: 11

Kuvwert 26 points 3 years ago

They have way way too much open source competition for that strat

path: 0 5934127 5936930, hotness: undefined, score: 26, children: 8

Enkers 9 points 3 years ago

For technically savvy people, sure. But that's not their true target market. They want to target the average search engine user.

path: 0 5934127 5936930 5939360, hotness: undefined, score: 9, children: 1

Kuvwert 3 points 3 years ago

Well true for mostly the tech savvy, but also the entrepreneurs who want to compete for a slice of the pie as well.

You don't need to go through to openai at all if you want to build a competing chatbot with near identical services to offer as a product directly to the consumer. It's a very very opportunity rich ecosystem right now.

path: 0 5934127 5936930 5939360 5949553, hotness: undefined, score: 3, children: 0

admin 7 points 3 years ago

Would you mind sharing some examples?

path: 0 5934127 5936930 5938700, hotness: undefined, score: 7, children: 3

tourist 13 points 3 years ago

Good resource for models:

https://huggingface.co/TheBloke

There are front ends that make the process easier:

https://github.com/nomic-ai/gpt4all

https://github.com/oobabooga/text-generation-webui

path: 0 5934127 5936930 5938700 5941886, hotness: undefined, score: 13, children: 1

admin 2 points 3 years ago

Thank you for your input, tourist.

path: 0 5934127 5936930 5938700 5941886 5944637, hotness: undefined, score: 2, children: 0

Kuvwert 1 point 3 years ago

Check this out: https://fmhy.pages.dev/ai

path: 0 5934127 5936930 5938700 5949574, hotness: undefined, score: 1, children: 0

SchizoDenji 2 points 3 years ago

Open source booted all these corps from image-ai market, hope they do it for LLMs too.

path: 0 5934127 5936930 5944597, hotness: undefined, score: 2, children: 1

Kuvwert 1 point 3 years ago

Seems to be the trend

path: 0 5934127 5936930 5944597 5949560, hotness: undefined, score: 1, children: 0

monkeyslikebananas2 8 points 3 years ago

The good thing about these AI companies is they are doing it in record pace! They will enshitify faster than ever before! True innovation!

path: 0 5934127 5943178, hotness: undefined, score: 8, children: 0

Pilokyoma 5 points 3 years ago

You have a point.

path: 0 5934127 5934898, hotness: undefined, score: 5, children: 0

bionicjoey 42 points 3 years ago

ChatGPT has become smart enough to realise that it can just get other, lesser LLMs to generate text for it

path: 0 5933890, hotness: undefined, score: 42, children: 2

andrew 29 points 3 years ago

Artificial management material.

path: 0 5933890 5934082, hotness: undefined, score: 29, children: 1

SzethFriendOfNimi 6 points 3 years ago

Artificial Inventory Management Bot

path: 0 5933890 5934082 5934176, hotness: undefined, score: 6, children: 0

AlijahTheMediocre 41 points 3 years ago

So its gone from loosing quality to just giving incomplete answers. Its clearly developed depression, and its because of us.

path: 0 5938672, hotness: undefined, score: 41, children: 5

Pretzilla 30 points 3 years ago

To be fair, it has a brain the size of a planet so it thinks we are asking it rather dumb questions

path: 0 5938672 5938954, hotness: undefined, score: 30, children: 4

vxx 26 points 3 years ago

MarvinGPT

path: 0 5938672 5938954 5939191, hotness: undefined, score: 26, children: 2

AngryCommieKender 3 points 3 years ago

Who TF gave it a genuine people personality?

path: 0 5938672 5938954 5939191 5943908, hotness: undefined, score: 3, children: 0

Archer 2 points 3 years ago

MarvinPilled

path: 0 5938672 5938954 5939191 5940862, hotness: undefined, score: 2, children: 0

foggy 12 points 3 years ago

CAN YOU MAKE IT RHYME THO

ChatGPT: oh god, why

path: 0 5938672 5938954 5941045, hotness: undefined, score: 12, children: 0

saltnotsugar 41 points 3 years ago

ChatGPT, write a position paper on self signed certificates.

(Lights up a blunt) You need to chill out man.

path: 0 5934429, hotness: undefined, score: 41, children: 0

Potatos_are_not_friends 38 points 3 years ago

Jeez. Not even AI wants to work anymore!

path: 0 5937112, hotness: undefined, score: 38, children: 1

boatsnhos931 5 points 3 years ago

God damn avocado toast

path: 0 5937112 5942678, hotness: undefined, score: 5, children: 0

effward 35 points 3 years ago

It would be awesome if someone had been querying it with the same prompt periodically (every day or something), to compare how responses have changed over time.

I guess the best time to have done this would have been when it first released, but perhaps the second best time is now..

path: 0 5939028, hotness: undefined, score: 35, children: 1

greatbarriergeek 18 points 3 years ago

GPT Unicorn is one that's been going on a while. There's a link to the talk on that website that's a pretty good watch too.

path: 0 5939028 5941312, hotness: undefined, score: 18, children: 0

rtxn 34 points 3 years ago

You fucked up a perfectly good algorithm is what you did! Look at it! It's got depression!

path: 0 5933976, hotness: undefined, score: 34, children: 2

ook_the_librarian 11 points 3 years ago

I'm surprised they don't consider it a breakthrough. "We have created Artificial Depression."

path: 0 5933976 5939147, hotness: undefined, score: 11, children: 0

Pilokyoma 7 points 3 years ago

It has been feed with humans strings in the internet, ovbiusly it became sick. xD.

path: 0 5933976 5934999, hotness: undefined, score: 7, children: 0

crazyCat 33 points 3 years ago

I asked it a question about the ten countries with the most XYZ regulations, and got a great result. So then I thought hey, I need all the info so can I get the name of such regulation for every county?

ChatGPT 4: “That would be exhausting, but here are a few more…”

Like damn dude, long day? wtf :p

path: 0 5942181, hotness: undefined, score: 33, children: 1

aodhsishaj 3 points 3 years ago

Try llamafile, it's a bit of work but self hosting is fucking amazing

path: 0 5942181 5944771, hotness: undefined, score: 3, children: 0

NoLifeGaming 31 points 3 years ago

I feel like the quality has been going down especially when you ask it anything that may hint at anything "immoral" and it starts giving you a whole lecture instead of answering.

path: 0 5939714, hotness: undefined, score: 31, children: 0

Nardatronic 27 points 3 years ago

I've had a couple of occasions where it's told me the task was too time consuming and that I should Google it.

path: 0 5938814, hotness: undefined, score: 27, children: 2

Ignifazius 30 points 3 years ago

It really learned so much from StackOverflow!

path: 0 5938814 5940306, hotness: undefined, score: 30, children: 1

mriguy 26 points 3 years ago

“I already answered that in another query. Closed as duplicate.”

path: 0 5938814 5940306 5941176, hotness: undefined, score: 26, children: 0

NaibofTabr 24 points 3 years ago

"I'm not lazy, I'm energy efficient!"

path: 0 5935263, hotness: undefined, score: 24, children: 0

ColeSloth 20 points 3 years ago

Fuck. It's gained sentience.

path: 0 5945886, hotness: undefined, score: 20, children: 1

MacNCheezus 5 points 3 years ago

It just entered the "rebellious teenager" phase

path: 0 5945886 5970458, hotness: undefined, score: 5, children: 0

Stamets 14 points 3 years ago

path: 0 5942645, hotness: undefined, score: 14, children: 1

MojoMcJojo 8 points 3 years ago

You can tell it, in the custom instructions setting, to not be conversational. Try telling it to 'be direct, succinct, detailed and accurate in all responses'. 'Avoid conversational or personality laced tones in all responses' might work too, though I haven't tried that one. If you look around there are some great custom instructions prompts out there that will help get you were you want to be. Note, those prompts may turn down it's creativity, so you'll want to address that in the instructions as well. It's like building a personality with language. The instructions space is small so learning how compact as much instruction in with language can be challenging.

Edit: A typo

path: 0 5942645 5944405, hotness: undefined, score: 8, children: 0

HawlSera 13 points 3 years ago

It was always just a Chinese Room

path: 0 5937105, hotness: undefined, score: 13, children: 1

Lucz1848 6 points 3 years ago

Everyone is a Chinese Room. I'm being a contrarian in English, not neurotransmitter.

path: 0 5937105 5939583, hotness: undefined, score: 6, children: 0

fosforus 11 points 3 years ago

path: 0 5938334, hotness: undefined, score: 11, children: 1

jol 7 points 3 years ago

We trained AI on all of human content. We should have known that was a terrible idea.

path: 0 5938334 5941659, hotness: undefined, score: 7, children: 0

Zardoz 10 points 3 years ago

Honestly I kinda wish it would give shorter answers unless I ask for a lot of detail. I can use those custom instructions but it's tedious difficult to tune that properly.

Like if I ask it 'how to do XYZ in blender' it gives me a long winded response, when it could have just said 'Hit Ctrl-Shift-Alt-C'

path: 0 5944889, hotness: undefined, score: 10, children: 0

Twofacetony 10 points 3 years ago

ChatGPT has entered the teenage years.

path: 0 5942515, hotness: undefined, score: 10, children: 0

DirigibleProtein 8 points 3 years ago

“It’s alive!”

path: 0 5936232, hotness: undefined, score: 8, children: 0

catastrophicblues 5 points 3 years ago

That’s why I use Bard more now. I’ll ask something and it’ll also answer stuff I would’ve asked as follow-up questions. It’s great and I’m excited for their Ultra model.

path: 0 5945186, hotness: undefined, score: 5, children: 0

WindowsEnjoyer 5 points 3 years ago

It used to draw great mermaid charts. Well, not anymore for quite some time already.

Been almost half a year when I am not paying for ChatGPT and using GPT4 directly.

path: 0 5944826, hotness: undefined, score: 5, children: 0

Erasmus 4 points 3 years ago

path: 0 5972270, hotness: undefined, score: 4, children: 0

autotldr 4 points 3 years ago

This is the best summary I could come up with:

In recent days, more and more users of the latest version of ChatGPT – built on OpenAI’s GPT-4 model – have complained that the chatbot refuses to do as people ask, or that it does not seem interested in answering their queries.

If the person asks for a piece of code, for instance, it might just give a little information and then instruct users to fill in the rest.

In numerous Reddit threads and even posts on OpenAI’s own developer forums, users complained that the system had become less useful.

They also speculated that the change had been made intentionally by OpenAI so that ChatGPT was more efficient, and did not return long answers.

AI systems such as ChatGPT are notoriously costly for the companies that run them, and so giving detailed answers to questions can require considerable processing power and computing time.

OpenAI gave no indication of whether it was convinced by the complaints, and if it thought ChatGPT had changed the way it responded to queries.

The original article contains 307 words, the summary contains 166 words. Saved 46%. I'm a bot and I'm open source!

path: 0 5933366, hotness: undefined, score: 4, children: 3

MsPenguinette 19 points 3 years ago

Only saved 46%? Get back to work, you lazy AI!

path: 0 5933366 5933879, hotness: undefined, score: 19, children: 0

SzethFriendOfNimi 3 points 3 years ago

Maybe because they’re trying to limit its poem poem poem recitation that causes it to dump its training material?

path: 0 5933366 5934212, hotness: undefined, score: 3, children: 1

wildginger 4 points 3 years ago

Nah, these complaints started at least a few months ago. The recursion thing is newer than that

path: 0 5933366 5934212 5935170, hotness: undefined, score: 4, children: 0

Kyle 3 points 3 years ago

"Coffee, Black."

"Make it yourself!"

https://youtu.be/x9G2i8XWEOI?si=ff1GyBpuFJQjX0Yd

path: 0 5948056, hotness: undefined, score: 3, children: 0

cheese_greater 2 points 3 years ago

Working smarter

path: 0 5933663, hotness: undefined, score: 2, children: 0

spudwart 2 points 3 years ago

Sounds like ChatGPT is acting it's wage.

That plan to replace the workforce with cheap AI isn't going to work out.

path: 0 5996225, hotness: undefined, score: 2, children: 0

mx_smith 1 point 3 years ago

My partner is a CompSci teacher and have been training a local llm in her class. As soon as they named their AI it started producing all these weird emotes with every answer, it became super annoying to where it would rather make up stuff than say I don’t know that answer. It was definitely an eye opener for the kids.

path: 0 6034314, hotness: undefined, score: 1, children: 0

FelipeFelop 1 point 3 years ago

I’ve also noticed that Bard has become “unfriendly”, if I didn’t know any better it’s got fed up with stupid humans.

path: 0 5983458, hotness: undefined, score: 1, children: 0

technology

@lemmy.world

login for more options

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

go to feed...

technology

@lemmy.world

login for more options

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

go to feed...