How bad are search results? Let's compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

3 years ago by Zen to c/technology

I find it pretty interesting that kagi is rated as Terrible search engine, even ChatGPT preforms better.

morrowind 55 points 3 years ago

This has been posted before, he has some weird criteria and no one has been able to replicate his kagi results

path: 0 7226371, hotness: undefined, score: 55, children: 0
firebyte 29 points 3 years ago

I was skeptical at first though let me tell you, Kagi is so much better. I get exact search terms, which is immensely useful as a programmer, rather than providing results for what Google thinks I want to search for. It's also really, really nice not seeing ads as search results anymore, ad blocker or no ad blocker.

Is it as comprehensive as Google search? It meets about 95-96% of my needs. I still use Google very infrequently for some really obscure domain specific searches if Kagi doesn't find anything useful, though that's getting rarer and rarer.

It's also easy to block AI generated sites that pop up providing just enough likeness, but really are regurgitated AI trash, or are 'Wikipedia clones'.

I have no financial interest in Kagi, other than paying to use it. It has certainly been worth it for me.

path: 0 7228650, hotness: undefined, score: 29, children: 7
Zen 7 points 3 years ago path: 0 7228650 7228801, hotness: undefined, score: 7, children: 5
qaz 1 point 3 years ago

Is it broken? It returns nothing.

path: 0 7228650 7228801 7233499, hotness: undefined, score: 1, children: 2
Zen 1 point 3 years ago

???

path: 0 7228650 7228801 7233499 7234289, hotness: undefined, score: 1, children: 1
qaz 1 point 3 years ago

It seems like the query was the problem. “Bitscan forward NASM” didn’t return anything but “NASM bsf” did return 2 results although they weren’t very useful.

path: 0 7228650 7228801 7233499 7234289 7237170, hotness: undefined, score: 1, children: 0
d13 -1 points 3 years ago

So this whole post is an advertisement, then.

path: 0 7228650 7228801 7231903, hotness: undefined, score: -1, children: 1
Urist 5 points 3 years ago

Personal recommendations are not really advertisements.

path: 0 7228650 7228801 7231903 7234715, hotness: undefined, score: 5, children: 0
plixel 4 points 3 years ago

I really liked Kagi at first, especially since I use it mainly for programming as well, but recently I feel like the quality has gone downhill. Right around the time they integrated the Brave stuff I've noticed a significant amount of me having to scroll down past the usual Google-like fluff results before getting to actually relevant information. It's a little sad to see because when I first used it, it was so good now it basically feels like a skinned Google-lite at this point. I'm still a customer but only because I haven't found a good alternative yet.

path: 0 7228650 7240922, hotness: undefined, score: 4, children: 0
filister 29 points 3 years ago

What a huge wall of text, so horribly formatted using a very ineligible font and it looks awful on the phone.

Plus it is kind of dumb to measure search engines based on such a low number of queries. You are introducing a huge bias and the generated search results are ranked by a single person also introducing his own bias. The idea is interesting the execution not so much.

path: 0 7230715, hotness: undefined, score: 29, children: 3
wurosh 10 points 3 years ago

Idk, doing this "properly" would take an immense amount of effort and manpower. This feels more like a "let me get enough info for an educated guess" EDA process, which still seems to have taken a lot of effort and I appreciate it a lot.

path: 0 7230715 7232634, hotness: undefined, score: 10, children: 2
akrot 1 point 3 years ago

But basing a recommendation on a ballpark anectdotal evidence is eidiculous.

path: 0 7230715 7232634 7251335, hotness: undefined, score: 1, children: 1
wurosh 1 point 3 years ago

Did he actually recommend one? That said, it's obvious the author favors Marginalia personally, but there's no point pretending they don't have biases. At least for me, making them obvious helps.

path: 0 7230715 7232634 7251335 7251966, hotness: undefined, score: 1, children: 0
podperson 19 points 3 years ago

Are people actually using ChatGPT as a search engine?

path: 0 7226253, hotness: undefined, score: 19, children: 12
MudMan 30 points 3 years ago

It depends. Chatbots are terrible at broad queries or parsing very detailed information, but they're surprisingly good with very fuzzy searches. If I want a link to a specific website I go to a search engine. If I want to ask "hey, what's that 80s horror comedy that's kinda like Gremlins but not Gremlins and it has one of the monsters coming out of the toilet in the poster?" I go to a chatbot.

EDIT: Heh. Just for laughs, I tried that exact query on Perplexity.ai. It got it right:

The movie you are referring to is "Ghoulies." It is a 1984 horror comedy film that features small, impish creatures similar to those in Gremlins. One of the iconic images associated with the movie is a Ghoulie coming out of a toilet, which is also featured on the poster.

path: 0 7226253 7227007, hotness: undefined, score: 30, children: 2
EldritchFeminity 8 points 3 years ago

I find this fascinating because that seems like the most difficult of the 3 to do for a normal search engine and sounds like an incredibly useful tool, but everybody and their mother seems to only care about whether it can do the other 2 or if you can trick it into spilling military secrets.

path: 0 7226253 7227007 7230040, hotness: undefined, score: 8, children: 1
MudMan 5 points 3 years ago

Well, yeah. So much of this conversation has gotten really dumb, with both advocates and detractors misrepresenting the tech and its capabilities and applying it to the wrong uses and applications as a result.

Honestly, early on I did think as a summary service for search queries it'd be more useful than it ended up being. It quickly became obvious that without the search results onscreen you basically have to fact check every piece of info you get, so it's only really useful to find answers you already know but had forgotten or that you need a source for.

But hey, at least I noticed that it kinda isn't before I built it as a key part of Windows. At this point if I was going to build a search app around this tech I'd use it for a short summary to replace Google's little blurb cards and still give you the raw results immediately below. It's only really good at parsing a wonky search prompt into a more accurate query. That's why when I have to use one of these I go to Perplexity instead of raw ChatGPT or Bing or whatever, it's the one that's built the most like that, although you still end up having to argue with it when it insists on being wrong and gets sidetracked by its own mistakes.

path: 0 7226253 7227007 7230040 7231028, hotness: undefined, score: 5, children: 0
nucleative 11 points 3 years ago

I use it as an answer engine. Queries like: what's that css property for xyz, or please summarize this email, or give me the top 25 most commonly used color words in English in a json schema like this.

All of that could be found with a normal search engine but I'd have to work harder and sort through a lot of trash along the way.

ChatGPT just understands what I'm looking for almost no matter how poorly worded my query is and just answers the question.

path: 0 7226253 7228400, hotness: undefined, score: 11, children: 1
lemmyvore 8 points 3 years ago

please summarize this email

As long as you realize they store and use that email.

path: 0 7226253 7228400 7232557, hotness: undefined, score: 8, children: 0
bleistift2 3 points 3 years ago path: 0 7226253 7227259, hotness: undefined, score: 3, children: 1
PipedLinkBot 2 points 3 years ago path: 0 7226253 7227259 7227269, hotness: undefined, score: 2, children: 0
SGG 3 points 3 years ago

It's a whatever question you ask engine. You can ask for the information directly and/or ask for sources to back it up.

path: 0 7226253 7227048, hotness: undefined, score: 3, children: 0
podperson 1 point 3 years ago

Interesting responses here - perhaps I’ll add it to my daily rotation as a science experiment.

path: 0 7226253 7227347, hotness: undefined, score: 1, children: 0
Imgonnatrythis 1 point 3 years ago

More and more yes. You often get an answer without having to link out to some bullshit site with ads everywhere. It's often pretty clean and precise.

path: 0 7226253 7229507, hotness: undefined, score: 1, children: 1
SpaceNoodle 0 points 3 years ago

Wrong, but precise.

path: 0 7226253 7229507 7229746, hotness: undefined, score: 0, children: 0
RagingInside 1 point 3 years ago
path: 0 7226253 7226746, hotness: undefined, score: 1, children: 0
Imgonnatrythis 15 points 3 years ago

What a painful read. Search results will vary to a degree based on your profession and interests. Personally I haven't found anything that beats Kagi and the UI is too notch which matters to me since I use search quite a bit.

path: 0 7229533, hotness: undefined, score: 15, children: 0
Aussiemandeus 15 points 3 years ago

I like duck duck go, but it is so hopeless when it comes to finding a local company website,

path: 0 7225951, hotness: undefined, score: 15, children: 1
SpaceNoodle 13 points 3 years ago

It's just Bing.

path: 0 7225951 7226438, hotness: undefined, score: 13, children: 0
qaz 14 points 3 years ago

The queries are:

download youtube videos

ad blocker

download firefox

Why do wider tires have better grip?

Why do they keep making cpu transistors smaller?

vancouver snow forecast winter 2023

A sample size of 6? This is useless.

path: 0 7233473, hotness: undefined, score: 14, children: 0
BentiGorlich 13 points 3 years ago

I am happy with kagi, so... But I abondoned google a long time ago and basically switched from DuckDuckGo to Kagi

path: 0 7227215, hotness: undefined, score: 13, children: 0
muntedcrocodile 13 points 3 years ago

Friendly reminder to go use searxng

path: 0 7229585, hotness: undefined, score: 13, children: 4
Tangent5280 6 points 3 years ago

Google just killed its cache service. Bad news for searxng.

path: 0 7229585 7229986, hotness: undefined, score: 6, children: 3
rigatti 12 points 3 years ago

How are they related? I have no idea how searxng works.

path: 0 7229585 7229986 7230429, hotness: undefined, score: 12, children: 1
Akasazh 2 points 3 years ago

Seconded

path: 0 7229585 7229986 7230429 7233607, hotness: undefined, score: 2, children: 0
muntedcrocodile 2 points 3 years ago

Someone will fix it and untill then i guess ill survive without google results in my searches

path: 0 7229585 7229986 7232834, hotness: undefined, score: 2, children: 0
maegul 5 points 3 years ago

Anyone else wonder if Dan Luu’s stuff is ever worth the read? Generally I’m interested in what he talks about and has to say, but every article/post of his gives me serious info dump vibes. And sure, I like deep dives and long form as much, even today, but I with his content I’m always feeling like I didn’t need to read all of this and that he just likes writing a lot. Anyone else? Not I didn’t bother reading this one because it definitely seemed not worth it.

path: 0 7227390, hotness: undefined, score: 5, children: 7
Aedis 5 points 3 years ago

At least what I see with this experiment/article is that is overly verbose, he takes a long time to get to the point. And then when he does his methodology shows an experiment that cannot be verified. Even when something is "subjective" we can still draw conclusions from it if we set up proper non-subjective ways of evaluating the results we see (ie. Rubrics). The fact that he doesn't really say what leads him to say in detail what is a "terrible/v. bad/bad/good result" is a massive red flag in his method.

After seeing that, I no longer read the rest of it. Any conclusions drawn from a flawed methodology are inherently fallacies or hearsay.

If in any case it is further explained in the article and that somehow refutes what I've postulated later on, then I would have to say that the article is poorly written.

All this to say... I agree with you, not worth the read.

path: 0 7227390 7229929, hotness: undefined, score: 5, children: 3
iopq 1 point 3 years ago

The entire post is exact details for why he decides each rating for each query

path: 0 7227390 7229929 7230466, hotness: undefined, score: 1, children: 2
Z4rK 1 point 3 years ago

No it isn’t. He for example evaluate that Kagi and Marginalia get the same score if you have to read as far down as to the 10th result for Kagi, while Marginalia has no answer. How is that the same score? There is no explanation. There is a lot of text, and then in the end he has made some subjective choices.

path: 0 7227390 7229929 7230466 7231248, hotness: undefined, score: 1, children: 1
iopq 1 point 3 years ago

Oh, then that's a bit inconsistent

path: 0 7227390 7229929 7230466 7231248 7249157, hotness: undefined, score: 1, children: 0
HenriVolney -3 points 3 years ago

Yet you bothered comprehensively shitting on his paper!

path: 0 7227390 7228812, hotness: undefined, score: -3, children: 1
maegul 5 points 3 years ago

I’m not shitting on it, just sharing my impression and arguably prejudice of his work and asking if anyone has shared or different perspectives. I’m very happy with the idea that his work is good and enjoyed by many (by all means he seems to have a healthily strong patreon following).

path: 0 7227390 7228812 7229353, hotness: undefined, score: 5, children: 0
HenriVolney -5 points 3 years ago

Yet you bothered comprehensively shitting on his paper!

path: 0 7227390 7228811, hotness: undefined, score: -5, children: 0
Telodzrum 4 points 3 years ago
path: 0 7239510, hotness: undefined, score: 4, children: 0
HenriVolney 3 points 3 years ago

Great read, thanks! On my way to try marginalia

path: 0 7228824, hotness: undefined, score: 3, children: 2
cyberpunk007 8 points 3 years ago

You fell for the marginalia ad!

path: 0 7228824 7229236, hotness: undefined, score: 8, children: 1
HenriVolney 2 points 3 years ago

Well, my bad. Anyway, I can't make it work with Firefox focus. The search results analysis is still relevant though. I miss the early days of light fast clean Internet search

path: 0 7228824 7229236 7231756, hotness: undefined, score: 2, children: 0
finthechat -8 points 3 years ago

I am both joking and being serious when I say this: the engine doesn't matter, it's a skill issue.

path: 0 7227571, hotness: undefined, score: -8, children: 4
Blue_Morpho 35 points 3 years ago

It's not 2010 anymore. Google has removed or ignores most regex that used to make it work well. It's not a skill issue. You can't make the queries you used to be able to do.

path: 0 7227571 7228571, hotness: undefined, score: 35, children: 0
bleistift2 14 points 3 years ago

Recently I saw a meme or post or tweet that I can’t find anymore. The gist was: In the 2000s, search engines were arcane tools which could only be used competently by few select people. Nowadays this has changed. Not because people have gotten smarter, but because search engines were dumbed down so much that it’s not possible to use them competently anymore.

path: 0 7227571 7228114, hotness: undefined, score: 14, children: 1
Plopp 4 points 3 years ago

Right? I feel the same about much of software design as well. Been a computer nerd my whole life and it's never been more difficult to use them competently than it currently is. Incredibly frustrating.

path: 0 7227571 7228114 7231973, hotness: undefined, score: 4, children: 0
Doolbs 1 point 3 years ago

I use Google a lot, and I don't have any problems finding relevant results. But, then again again I do a lot of site searches: site:edu, site: gov. Or, even narrowing down more doing a site search like: site:jstor.org; a lot better than being SEO blinded by doing a general Google search.___

path: 0 7227571 7227781, hotness: undefined, score: 1, children: 0
glowie -13 points 3 years ago

No Brave?

path: 0 7225402, hotness: undefined, score: -13, children: 0
technology
technology

@lemmy.world

login for more options
87359
21246
15760

This is a most excellent place for technology news and articles.

Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


go to feed...