Windows 11 will soon be able to describe images on your screen using AI — and it'll all be done locally

a year ago by moe90 to c/technology

A new "Describe Image" feature is coming soon to Click To Do, which will allow Windows 11 to use AI to describe an image that's currently on your screen.
napkin2020 21 points a year ago

Multimodal models has a lot of potentials in terms of accessibility.

But fuck Microsoft. you're not fooling anyone.

path: 0 18248041, hotness: undefined, score: 21, children: 0
noretus 19 points a year ago

Just had a chat with a gamer friend who was ditching Windows for good and switching to https://bazzite.gg/. I'll probably do so as well. Apparently I've been needlessly scared of Linux and I had missed that SteamDeck encourages devs to make games work for Linux.

path: 0 18252761, hotness: undefined, score: 19, children: 4
BassTurd 13 points a year ago

If you aren't playing a game that has kernel mode anti cheat, like League of Legends, then your gaming experience will probably be the same if not better.

path: 0 18252761 18253904, hotness: undefined, score: 13, children: 0
SoleInvictus 8 points a year ago

Linux has come SO far. I first tried it about a decade ago, wrecked my install, and came crawling back to Windows. A few months back, I installed Ubuntu (tease all you want, peanut gallery, it's better than Windows) on my laptop and it's so much better. It just works great, I have my system customized to how I want it to work, and the poor CPU fan is no longer blasting even when the system is idle. My battery life also gained almost an hour.

path: 0 18252761 18256018, hotness: undefined, score: 8, children: 1
Theoriginalthon 5 points a year ago

I've been using Ubuntu since 7.04, it's all Linux underneath just with a slightly different look

path: 0 18252761 18256018 18257374, hotness: undefined, score: 5, children: 0
Toribor 8 points a year ago

Bazzite is great.

path: 0 18252761 18259153, hotness: undefined, score: 8, children: 0
DaddleDew 14 points a year ago

An image is worth a thousand words. How is reading a text describing what is on the screen going to be better than just looking at the screen yourself, something you'll need to do to read the description anyway? Aside from accessibility for the blind, the practicality such a technology is questionable.

The motivation behind this is obviously to facilitate the collection and reporting user profiling data. Accessibility for the blind is only a side effect. Tech companies have been doing it with automated audio transcriptions for years already, now they're after what you look at on your screen.

path: 0 18247991, hotness: undefined, score: 14, children: 4
SheeEttin 11 points a year ago

Blind people exist

path: 0 18247991 18250482, hotness: undefined, score: 11, children: 1
DaddleDew 2 points a year ago

Read the whole post. I acknowledged them already and am expressing my doubts over the true motivations that drive Microsoft to force a tech like this upon all their users and express my concerns over the real use they will make of this technology.

Don't you try to change the meaning of my post just so you can have a cause to white knight over. This isn't Reddit.

path: 0 18247991 18250482 18251504, hotness: undefined, score: 2, children: 0
SSUPII 6 points a year ago

Sometimes things are unlabeled, and this is for that

path: 0 18247991 18248445, hotness: undefined, score: 6, children: 0
rebelsimile 2 points a year ago

this is 100% right, you don’t need an AI to describe something you’re already looking at. This is an absurd feature (again aside from the accessibility portion but that’s not what this is).

path: 0 18247991 18250341, hotness: undefined, score: 2, children: 0
iMastari 13 points a year ago

Why would I, or anyone want this?

path: 0 18254469, hotness: undefined, score: 13, children: 3
SoleInvictus 11 points a year ago

The blind or vision impaired would appreciate this. Past that, hard pass.

path: 0 18254469 18255926, hotness: undefined, score: 11, children: 0
shortwavesurfer 3 points a year ago

I use a screen reader due to being blind, and this would be incredibly helpful. Mind you, I sure as hell wouldn't use Windows in order to get it. But if I had the option to do this on Linux, I would totally go for it.

path: 0 18254469 18264163, hotness: undefined, score: 3, children: 0
lka1988 2 points a year ago

My wife, when she's not wearing her prosthetic corneal lenses.

path: 0 18254469 18274005, hotness: undefined, score: 2, children: 0
cecilkorik 6 points a year ago

Never been happier that all my computers run Windows 10 or Linux. Windows 11 is dead to me, and if anything happens to accidentally get it installed somehow, it's going to be replaced with Linux going forward.

path: 0 18249395, hotness: undefined, score: 6, children: 0
FelixCress 5 points a year ago

Go and fuck yourselves.

path: 0 18250810, hotness: undefined, score: 5, children: 0
rdri 5 points a year ago

A small conversation with Copilot makes my laptop choke on RAM, and that stuff is processed on cloud. No way I will allow Microsoft to run an AI locally.

path: 0 18259579, hotness: undefined, score: 5, children: 0
DrDystopia 4 points a year ago

I never cease to be impressed by the continual enshittification of Microsoft products, and the bar was pretty low to begin with.

path: 0 18253925, hotness: undefined, score: 4, children: 0
Blueberrydreamer 3 points a year ago

Only available on Copilot+ PCs

Oh good, I don't have to worry about figuring out how to disable this.

path: 0 18249805, hotness: undefined, score: 3, children: 2
Grostleton 2 points a year ago

*Yet

The Copilot+ moniker is only limited to a specific subset of PCs for now until all hardware on the market meets the requirements, then it's inescapable. Microsoft will have your data, and I'm all but certain of an ulterior motive beyond just that for forcing this screen capture garbage onto their userbase.

Better to prepare now and move off the MS ecosystem entirely.

path: 0 18249805 18268769, hotness: undefined, score: 2, children: 1
Blueberrydreamer 1 point a year ago

Thankfully I already have for the most part. Unfortunately I'm stuck dual booting until I can find a way to run fusion 360 on Mint.

path: 0 18249805 18268769 18269737, hotness: undefined, score: 1, children: 0
resipsaloquitur 3 points a year ago

Why.

path: 0 18249868, hotness: undefined, score: 3, children: 0
rottingleaf 2 points a year ago

Trust us bro.

path: 0 18249994, hotness: undefined, score: 2, children: 0
Zwuzelmaus -1 points a year ago path: 0 18248461, hotness: undefined, score: -1, children: 0
technology
technology

@lemmy.world

login for more options
88797
22181
14530

This is a most excellent place for technology news and articles.

Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


go to feed...