The Fediverse ALT-Text issue

2 months ago by MadeInDex 📰🌎 to c/fediverse

cross-posted from: https://lemmy.world/post/45437770

I've been thinking a lot about the Fediverse ALT issue.

Some people are annoyed by posts without Alttext, & others get reminded¹ to add it.

The core question is: How can we improve accessibility?

Proposal: ☑️ Add a user filter to hide media posts without AltTag ☑️ Reduced engagement on hidden posts would encourage adding Alt text ☑️ People who need accessibility wouldn't have to encounter unlabeled media

If this gets traction, I'll open a Mastodon GitHub issue (maybe on others too?).

¹ https://mastodon.social/...

beveradb 21 points 2 months ago

These days, it would probably be more effective to make it easy for instance operators hook up a local multi modal llm to the server to auto generate alt text for all images posted on their instance. I'd happily donate to cover the infrastructure costs for any instance owner who was doing that

path: 0 23151196, hotness: undefined, score: 21, children: 15

madeindex 13 points 2 months ago

Definitely more effective than every disabled person etc. having to generate it! Have you seen the Mastodon image text to alt text feature? I guess they could simply expand on that?

path: 0 23151196 23151293, hotness: undefined, score: 13, children: 0

Cris_Citrus 4 points 2 months ago

While I understand the potential value of that, it is worth also acknowledging that ai will never understand the human, social, contextual meaning of things its providing alt text for, and theres a fair argument to be made that folks with disabilities deserve those things

Id welcome perspectives from folks with relevant disabilities who use or depend on alt text :)

path: 0 23151196 23158986, hotness: undefined, score: 4, children: 3

Sproid 1 point 16 days ago

They may deserve it but between No alt text and good-to-passable alt text, 🤷‍♂️

path: 0 23151196 23158986 24131534, hotness: undefined, score: 1, children: 0

SlurpingPus 1 point 2 months ago

‘iPhone Magnifier and Voice over’

I don't hear her complaining.

path: 0 23151196 23158986 23166632, hotness: undefined, score: 1, children: 1

Cris_Citrus 2 points 2 months ago

Yep, which is why I acknowledge that thats not my ingroup and while I think we should be considerate of potential issues with ai as a accessibility solution, its important to listen to people who actually depend on things like alt text. I've heard from people who are frustrated that ai gets treated as a standin for other humans caring about accessibility and find it unable to fill their needs, and I'm sure there are also people who find it really helpful. I couldn't claim to know what perspective is more common among folks with visual disabilities 🤷🏻‍♂️

But I do think its worth considering and not just saying "ai solves this problem, let's just give visually impaired people ai descriptions" without trying to discuss what that means for people and listen to folks on whether thats what's needed or wanted

path: 0 23151196 23158986 23166632 23186910, hotness: undefined, score: 2, children: 0

lambisio -1 points 2 months ago

I thought we were anti AI...?

path: 0 23151196 23156761, hotness: undefined, score: -1, children: 9

SlurpingPus 7 points 2 months ago

Take a look at how blind people can use phones to learn what's around them, and tell me afterwards that you're still against AI.

path: 0 23151196 23156761 23159851, hotness: undefined, score: 7, children: 0

beveradb 3 points 2 months ago

It's a polarised topic but doesn't need to be, IMO - I still see it as a tool, albeit a powerful one with lots of risks and drawbacks, but the cats out of the bag at this point so we may as well roll with it and make the most of the applications for good. Like with many other massive, rapid technical advancements of the past.

path: 0 23151196 23156761 23156959, hotness: undefined, score: 3, children: 6

lambisio -4 points 2 months ago

AI is quite unlike other "rapid technical advancements of the past" in that sense. It's so damaging to people, society and environment that there is currently no possibility that any use you make of it, "well-intentioned" or not, can improve things. Any "we might as well use it" s little more than joining the oppression.

path: 0 23151196 23156761 23156959 23163909, hotness: undefined, score: -4, children: 5

p03locke 7 points 2 months ago

Imagine thinking that LLMs are so toxic that you'd think we're splitting the atom, a technology so polarizing that it became a security issue. Any yet, nuclear power is an incredible scientific achievement, and something we still use all over the place. But, it's also a technology that we still aren't fully utilizing 80 years later, because of fearmongers like you taking away any sense of nuance in the conversation, and letting blind absolutism govern your opinions.

path: 0 23151196 23156761 23156959 23163909 23164231, hotness: undefined, score: 7, children: 0

SlurpingPus 5 points 2 months ago

I replied to you with an example of AI recognizing things that the phone camera sees, to tell the blind user about them. That reply was before this comment of yours. I myself am also a person using auto-generated subtitles on YouTube, because my hearing isn't too good, especially for the non-native English language.

Yet you claim that none of these uses improve things. So you see less-able people as unworthy of improvements to their lives. Nice to know, because an opinion stemming from bigotry can be summarily dismissed.

path: 0 23151196 23156761 23156959 23163909 23166579, hotness: undefined, score: 5, children: 3

MysticMushroom1776 1 point 2 months ago

Speak for yourself, being able to describe an image or generate subtitles when there are none made by the original creator while it might not be perfect, is great for accessibility and people who have hearing or visual impairments are happy to have them.

Also because a lot of anti-AI people on here have decided that impersonating me, making fun of my cancer an alcoholism, and even straight up doxxing me is an acceptable reaction to me hosting AI friendly communities is an acceptable reaction I don't have any more patience for these people, especially considering that 90% of their arguments are debunked by on-device AI, and the other 10% are neoliberal Pro-IP and Pro-Copyright arguments.

path: 0 23151196 23156761 23200352, hotness: undefined, score: 1, children: 0

lambisio 12 points 2 months ago

Why not start with autotagging image posts with no alt text with something like Mastodon's #Alt4Me? It summons the human element to help submit the data, instead of depending on AIs that we all know the stupidity and evil they do.

path: 0 23156812, hotness: undefined, score: 12, children: 3

Snoopy 4 points 2 months ago

That's a good start but i'm worried that most of us are lazy. They won't change their habits unless you enforce it.

I wouldn't condemn AI in this case. For example most of captioned video come from AI. Yes there is typo, lenghty text, missing text, nonsensical text but they do a fairly good job. No one have the time to create subtitle, and engaging professional may be costly.

Without AI, i wouldn't be able to watch and understand videos, podcasts nor get a transcript.

path: 0 23156812 23158838, hotness: undefined, score: 4, children: 2

lambisio 1 point 2 months ago

No one have the time to create subtitle, and engaging professional may be costly.

Bitch I lived through the anime fansub times of the early 00s. Sometimes the fans had the subs ready before the episode came out, and they were top notch quality, even with localization considerations. And they were doing it for free (if still for the cred). AI can only ever hope to have that sort of dedication.

Like, really, what kind of brainrot do you have? We've had internationalized captions and content decades before we had AI.

path: 0 23156812 23158838 23163934, hotness: undefined, score: 1, children: 1

Snoopy 2 points 2 months ago

TL; DR

However, not every videos are captioned. You can't match the scale of created content.

I have a nuanced opinion on AI. They are part of corpothieve, tech bro that destroy our environement and promote fascism or occidental visio on the world. On the other hand, it is also a tool that can help deaf people in their daily life and scientific research.

Brainrot

I didn't insult you. I shared my deaf experience. We come from different background...was it needed ? Why ?

I was also there in 00s. Yes, my mom also created subtitle for some french movies. I also created some subtitle for video, translated them...Yes, there was fansub, with a huge dedicated communities. They are awesome and they played a keypart on sharing other culture. Thank you to you all and them. 😊

the scale of uncaptionned videos

However, not every videos are captioned.

That's the point you misunderstood : the scale of uncaptionned video and the deadline. Given the huge amount of videos, a fansub can't match the scale of video content created every days. That's where AI can save my day. It does help me.

If i watch something that doesn't have a fansub, community : 0 captions. So i have to create a request in your communities. some people may accept, some people may reject it. At best i have to wait some days or months. It depend on our free time since we are doing voluntary work.

You omited podcasts. If i try any french webradio : 0 transcript. Some people used AI to get a transcript for their podcast. They reread and corrected it. It helped them getting a transcript quickly and i benefit from it.

my deaf experience

I studied space design. Teachers shared lot videos on architecture, those videos weren't captioned, so as a student i poured lot effort to understand what they said in those videos...At the end of day, i was exhausted. I poured too much effort and ended up in burn out.

Later, i followed a course to become a farmer. It was during the Covid period. I had to wait 6 months to get sign language interpreters. Not only i lost half a year battling to get sign language interpreters, but i also had to watch video on agroecology without manmade captions and participate to visio meeting...Luckily, there was some apps as Ava, google transcript and later Lokas (free software from framasoft. It use whysper) that use artificial intelligence. Teachers and students weared mask, i couldn't read on their lips. So how do you handle it ?

I can't imagine how i would deal it without those AI app. It would be hell and would have stopped my lesson because of burn out.

The french election in 2024

Another example. In 2024, the french president dissolved the national assembly. Election were upcoming in less a month. I battled very hard to get captionned videos, contacted lot people, joined various group, called deputies to get their video captioned for deaf people and translated in sign language. We were a team and only got two or three videos...

conclusion

I can use those AI app when :

no one is taking note on a collaborative pad
if interpreters aren't available.
if there is no caption

Lastly, some deaf friends told me they used AI to help them writting good sentence. Most of deaf people struggle with writting.

So, I have a nuanced opinion on AI. They are part of corpothieve, tech bro that destroy our environement and promote fascism or occidental visio on the world. On the other hand, it is also a tool that can help deaf people in their daily life and scientific research.

path: 0 23156812 23158838 23163934 23166081, hotness: undefined, score: 2, children: 0

northernlights 6 points 2 months ago

Add a little input box after clicking submit when things like alt text are missing? Seems like an easy problem.

path: 0 23155156, hotness: undefined, score: 6, children: 0

nasi_goreng 6 points 2 months ago

Some software does not support alt-text, so people write the text on the post itself or comment.

path: 0 23160502, hotness: undefined, score: 6, children: 2

Snoopy 2 points 2 months ago

Well that's what i did on lemmy in the begginning.

So a middle ground is need or maybe we should work on a fediverse accessibility charter ? Something that set some rules for the whole fediverse :

new software
3 years software
various disabilities use case
standarized colors scheme and font. So it would be easier to update them.

path: 0 23160502 23160543, hotness: undefined, score: 2, children: 1

nasi_goreng 2 points 2 months ago

Some fedi software are esoteric, like Misskey have Misskey flavoured markdown (example), even Mastodon cannot display the feature. There are fedi software that can be accessed via console, zero image displayed. Some others have non-conventional design that caters to specific use, basically not even for mainstream people, so they don't even follow any accessibility rules.

path: 0 23160502 23160543 23161245, hotness: undefined, score: 2, children: 0

j4k3 6 points 2 months ago

OCR tool+ to autogen a suggested alt text. The path of least resistance needs to be lowered.

Alternatively, inverting the paradigm is likely to cause less issues and push back. Add the automated tool the the end user in need of the version. This obviously creates the issue of data quality and trust, but for the smaller group. What if there was a reply field silently posted to everyone's notifications feed indicating anonymous instances of the tool being used to fill in the gaps for alt text? The message would need to be opt out or carefully presented. Perhaps it could be possible to modify the post itself via the tool? Better yet, make the alt text field a Wikipedia style affair anyone with an account can edit, but with a lock available to the OP. That would create much more healthy awareness of the need for alt text, as people posting the content will see the places where gaps are filled by an automated tool. It gives them the chance to edit. This does little to initially improve the experience of the most active alt text users, but it creates a strong cultural shift in awareness that should improve the situation greatly in the long term IMO.

path: 0 23151234, hotness: undefined, score: 6, children: 0

Kolanaki 4 points 2 months ago

There is already a not-hidden field presented when making a post to fill in alt-text on Lemmy, but I wonder how many users know its purpose. It assumes everyone knows what it is. It would be helpful if it had a short explination next to the input field, with a link to a larger explination and why it should be used.

And for images that are just screenshots of tweets or are otherwise just text, maybe some kind of image to text process can be done automatically to fill the alt-text out. 🤷‍♂️

path: 0 23150853, hotness: undefined, score: 4, children: 3

madeindex 3 points 2 months ago

I guess even a questionmark (with a link to a popular website explaining it) next to the alt-field would suffice.

Mastodon actually already has that, but manually! :)

path: 0 23150853 23151246, hotness: undefined, score: 3, children: 2

frischkaesbagett 5 points 2 months ago

I created a github issue for alt Text warn before post and the developer immediatly started to work and it seems like he improved it really fast: https://github.com/LemmyNet/lemmy-ui/pull/4021

10/10 can recommend would do it again

Edit: added rating

path: 0 23150853 23151246 23151338, hotness: undefined, score: 5, children: 1

madeindex 2 points 2 months ago

wow that is fast!!! Mastodon is definitely slower in this particular case! Somebody sent me a similar ticket from 2024 ;)

path: 0 23150853 23151246 23151338 23151372, hotness: undefined, score: 2, children: 0

Snoopy 3 points 2 months ago

A simple one, i believe that softwares must enforce them rather letting users doing it.

Here is some ideas :

refuse "image post" without alt text.
when you upload picture, you must provide their alt text, if there is no alt text, the picture will be deleted.
hide user post based on alt text and remove post score if the alt text is bad.
deactivate crosspost on posts who don't have alt text

It maybe harsh but i believe that's the only way to reach accessibility.

Edit :

alt text autogen + a reminder bot will help users
The app fedilab (mastodon) has some good rules on post with alt text.
collaborative alt text.

path: 0 23153171, hotness: undefined, score: 3, children: 4

forestbeasts 8 points 2 months ago

That's a really good way to end up with fake meaningless "alt text" that's only put there to get around the restrictions.

Caring about meaningful image descriptions is a cultural thing.

-- Frost

path: 0 23153171 23156046, hotness: undefined, score: 8, children: 1

Draconic_NEO 1 point 2 months ago

I agree, I've already seen people on Mastodon doing this. This isn't the solution.

path: 0 23153171 23156046 23168394, hotness: undefined, score: 1, children: 0

lambisio 6 points 2 months ago

refuse “image post” without alt text.

Mostly* works, but increases significantly the barrier of entry to engage in "conversational" / "collquial" fediversing, and subjects the largest groups of people to the limitations of their tooling.

when you upload picture, you must provide their alt text, if there is no alt text, the picture will be deleted.

I assume this is for when you attach a picture to an otherwise text / link post? In that case, I think this is a very good path, yes, since in this ase the user who is adding the picture is already engaging in the extra work of getting it.

hide user post based on alt text and remove post score if the alt text is bad.

No CCP style shit please, we already have enough of that with Piefed.

Adding posts with no alt text to a predefined category or tag (such as Mastodon's #Alt4Me) sounds like a reasonable alternative.

path: 0 23153171 23156803, hotness: undefined, score: 6, children: 1

Snoopy 1 point 2 months ago

Adding posts with no alt text to a predefined category or tag (such as Mastodon’s #Alt4Me) sounds like a reasonable alternative.

Yes you are right. :)

No CCP style shit please, we already have enough of that with Piefed

I don't see why we should support content without alt text. There is people with various disability

I'm offering to void the post visibility :

the post won't receive upvote, so it won't reach the top of your timeline
user won't be able to crosspost them.

I'm deaf and i think we should all do an effort to make the web accessible. Everyday, i have to adapt to your world.

People can give a small fraction of their time to improve accessibility. But you are right it maybe too harsh, we should find a middle ground that invite users making the web a place place rather them punishing them.

**Long explaination on piefed score**

There is pro and cons using lemmy or piefed. Both have a different stance on moderation and offer a different experience.

Lemmy doesn't check downvote or users with bad reputation. How are you able to act with toxic users ? For example one downvoting a lot and the other one that post terrible thing thus get lot downvote ? Is this good for your community ? I don't think you will enjoy interacting with them.

Piefed score was requested by most admins and mods. It help us creating good community, our main job is creating a place where people enjoy sharing their experience. There are 2 scores :

attitude : it calculate the ratio of downvote/upvote. How do you react when people have lot negativity ? Most of us are around 70%. If you downvote a lot, that's not a good behaviour. There are few users or bots that downvote a lot. their attitude score is around 20% something like that, it is low. It removes the ability to downvote until you upvote and reach a threshold. So it invite user to use other tools (filter) to improve their timeline instead of voting.
reputation : that's the upvote you receive. Posting meme don't boost your reputation. Why ? Because of karma farming. Let's say i post terrible things, and i receive lot downvote. Meanwhile, i post fun meme and get lot upvotes...

So it just give us a general idea on users and improve community ambiance.

Futhermore, you can't just take the scoring system alone. Those features work together :

mods can remove downvote per community. I removed them in my communities.
filters : you can filter keywords, communities, instances and users.
report. It is more efficient, i prefer it over the automod that work well.

path: 0 23153171 23156803 23158050, hotness: undefined, score: 1, children: 0

kbal 0 points 2 months ago

path: 0 23151573, hotness: undefined, score: 0, children: 0

fediverse

@lemmy.world

login for more options

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, Mbin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Posts must be on topic.
Be respectful of others.
Cite the sources used for graphs and other statistics.
Follow the general Lemmy.world rules.

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)

go to feed...

fediverse

@lemmy.world

login for more options

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, Mbin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Posts must be on topic.
Be respectful of others.
Cite the sources used for graphs and other statistics.
Follow the general Lemmy.world rules.

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)

go to feed...