I'm working on a self-hosted search service called Hister with the same goal when I started Searx development: reduce dependence on online search engines.

Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. It provides a flexible web (and terminal) search interface & query language to explore saved content with ease or quickly fall back to traditional search engines. This is a fundamentally different approach than what Searx follows and solves most of the weaknesses of metasearch engines. Of course it has its own weaknesses as well, but most of these are not conceptual and can be resolved by improving the software (and datasets)

I've been using it for a few months and as my local index is growing I can avoid relying on external search engines - and even websites listed in results - more and more frequently.

The initial reception is overwhelmingly positive with already more than 30 contributors and hundreds of contributions. Currently it can help with "recall" type searches mainly, but I'm planning to provide pre-indexed thematic datasets and I'm drafting a peer-to-peer index sharing concept. Maybe you can find it useful as well (or at least have some constructive criticism =]).

Links: - https://hister.org/ - https://github.com/asciimoo/hister - Background/motivation/beginnings: https://hister.org/posts/how-i-cut-my-google-search-dependen... - Small read-only demo: https://demo.hister.org/

I would say that for almost all of my searches the AI Overview feature contains exactly the answer I was looking for, and I don’t even have to leave Google to get it. It’s been a very positive addition.

We live in the dystopia we deserve. We have built it with our own hands and it is here to stay.

[1]: https://kagi.com

[2]: https://uruky.com

Normal people are using AI for search more already, Google is just trying to stop their primary business from completely disappearing.

The UI has gotten a bit clunkier over the years, but it's still good, still more focused than Google's.

As I write this, I give Google Search a quick try and notice that the first thing you see is a full-screen cookie banner!? On my laptop, I even have to scroll to reach the Reject/Accept buttons, and keyboard controls don't work at all. I can't believe people still use this crap.

No thanks, I like ads blocked.

https://web.archive.org/web/20260523024427/https://techcrunc...

I was quite fond of ixquick but it shut down ages ago. These days I like etools.ch especially since it includes results from search engines like marginalia etc that I tend to forget to search directly but like having meshed into my general searches. Plus you can change which engines it uses in settings and it shows which search engine(s) each link came from which is handy.

On a related note I like to check out Serdys list of search engines with their own indexes once if a while. It gets updated here and there and includes a fair amount of search engines I don't tend to see elsewhere.

https://seirdy.one/2021/03/10/search-engines-with-own-indexe...

This is pure observation/anecdotal. I have no measurements to back this up but I think others will share this view.

The idea is simple: collect search stubs, short documents with routing information about a topic. When you want to search something your agent finds the right stub and looks up where on the web to go find that information. Your information is always fresh, while a search stub changes slowly, over years, because the entry points tend to be more stable.

If we use a 4B local model and another 4GB for search stubs it could be portable enough to download and install. As you use it you also generate your own search stubs on top of the generic package. A stub could contain links to news feeds, high quality hubs, search engines and of course, actual websites. High quality search stubs can be generated with any frontier LLM piggy backing on its agentic search capabilities.

I think the idea of managing a collection of search stubs as a replacement for centralized search engines is important because it would wean us of one of the last centralized points of the internet. Google plays many ranking games on top of users and publishers, serving their own interests first. I want out of that arrangement.

local LLM + local stub index + proxy == anonymity

They intercept a non-trivial portion of the web's traffic and presumably are as equipped as any other company is to build a good index. They're also the only the only company that has an interesting alternative incentive structure for creators.

Google still maintains a web search mode that's free of AI overviews/chat exhortations (as well as ads, if you use an ad blocker). https://www.google.com/search?q=foo&udm=14 is the format of the search URL, and tenbluelinks has instructions on how to use it as your default engine on various platforms.

That said, I've stopped using this as a founder. While I personally like the web search results more (if I wanted synthesis of results, I'd use dedicated agentic-loop-capable tools that are a hotkey away), it's far more important to understand (and empathize with) our users' experiences, good and bad, when they use Google in its full AI extravagance in practice.

I think the reason is my searches are almost entirely long-tail searches that Kagi's index just isn't good enough for. I am never searching for something like "best mattress" or anything else that is heavily SEO'd - it's always something very specific - so the result page in Google looks pretty much exactly like the Kagi page, only it nearly always has the result I'm looking for where Kagi's doesn't.

The current catalog covers 100+ providers, 1000+ universities, and 250,000+ courses.

Originally a weekend side project that I first shared with the world here on HN itself on Nov 29, 2011 [1]. Currently bootstrapped and profitable.

https://news.ycombinator.com/item?id=3288775#3289393

I've been using Ddg and Brave for general search and Yandex for deep-ocean expeditions (of the jack sparrow variety) and topics that US tech giants censor. I am looking for good Chinese search engine so that I can search things that Yandex censors or when Yandex's bot detection goes crazy (I get blocked with infinite captcha about 25% of the time).

The only thing Google remains good at is local search. If I want to buy something locally, nothing else comes close.

There’s so much content getting buried now.

If you’re looking for anything remotely niche or legally gray, like sports streams or ebooks, you’re often better off using Yandex or you’ll never find it.

The old Google search engine that used to properly index and surface the open web has been gone for a long time.

https://kagi.com/

I'd guess due to compute constraints, AI overview will struggle to reach truly great quality. That said, for now I find adding this section at the top still useful to me. The broader decline in Google's search quality is the bigger drag on me.

Google hasn't been "Google" for quite some time.

* I use Kagi and DuckDuckGo before that.

The last use case I had for google was Google Scholar, but it now appears to block anyone who blocks google tracking. But this is where ChatGPT does an excellent job of generating lists of technical papers and reviews and it interprets natural language queries with no problems. The kind of complex logical search queries google used to support (what, 15 years ago?) can be written without strict logical language (! & | ()) and all that. Pubmed isn’t bad for cross-checking and simple searches. And if you put sci-hub into the yandex.com/search box and click on yandex ai it tends to tell you where the current active sci-hub sites are, which is handy.

It incorporates results from both Google and Bing: https://support.startpage.com/hc/en-us/articles/452243553384...

Google and others are racing to integrate AI but - for me - there are two hidden costs. 1. Access to the web becomes mediated and monetised even more. 2. Energy consumption up to 100x normal searches has an environmental impact.

I would rather not.

Edit: clarifying that this is not strictly due to ads. I think the article itself is an ad judging by the slug 'six-search-engines-worth-trying-now-that-google....'. Usually such articles include a plug. I am not disabling adblocker to read that plug

I can't say I've noticed any changes about google search on desktop recently. Yes; there is an AI overview widget at the top of the page; but it's been there for at least a year.

Has anything changed about Google search results for you?

Maybe this is a frog in boiling water situation but… it’s just the same search as it’s always been, there’s just now a chunk of Gemini up top, that you take with a grain of salt, same way you’d take the promoted results. If you don’t like it, Adblock it.

When I ask myself honestly, has google search gotten worse over the past 25+ years? My answer is… ehhhhhh… not really?

Other people have always claimed google was ‘getting worse’ and I just don’t see why it should be any more true now than it was then. Isn’t this just the latest round of whinging?

  - Organize the world's information
  
  - Don't be evil

From https://en.wikipedia.org/wiki/Don%27t_be_evil :

> Between 21 April and 4 May 2018, Google removed the motto from the preface, leaving a mention in the final line: "And remember... don't be evil, and if you see something that you think isn't right – speak up!"

That sounds like NY to me.

I remember the first time I copy-pasted something off a major website, in quotes, because I wanted to follow up on it, and got no search results. Not even from the site I copy-pasted from. "Search" has been in name only for a very long time.

Build directories again.

This is going to happen whether Google does it or not. The toothpaste is out of the tube.

you can simply set the search as you like and save as a search url [1] that can be set in the browser once and forget.

their bangs are useful to search directly in a specific site (e.g. useful workaround for reddit search results not coming anymore). when i tried kagi, i noticed that they have adopted a similar system, which has to be inspired.

my overall search experience has been poor even before the GPTs, but this is just fine for almost everything.

[1] https://duckduckgo.com/duckduckgo-help-pages/settings/params

I'm afraid of them selling out at this point. If they go, I honestly don't know who will go in their place.

I changed my default search engine to: https://www.google.com/search?q=%s&udm=14

Author should mention that you can change your browser's default search engine.

Before, search engines provided links, links provided the info we were looking for. Now AI provides that info without the middle man, links are just a footnote in case you need them.

I use it through OpenRouter - I love how the pricing is per search and isn't a subscription.

https://www.mojeek.com/about/why-mojeek

The three dominant English search engines with their own indexes are Google, Bing, and Yandex but this list has many spiders and engines that traverse the web.

I think Google is getting its act back together.

Seeing as Apple has reportedly partnered with Google to power their LLM offering, I don’t expect that to change soon. I hope I am wrong.

      &udm=14

Different people/bots scrape the net and add it to a distributed database optimized for search.

Each query could cost a crypto micropayment to avoid DDoS. Or maybe a slightly larger payment to download the whole database so you can use it privately or create a competing centralized or decentralized search.

Yes, we hate crypto, but it seems useful here. It's bad if 1 entity can gatekeep both the database and access to it, no matter how non-evil they seem now.

We might even index torrents, use speech-to-text for music, movies, video clips and other things like that. So you'll search for a phrase from a movie and it will be there even though no one mentions it on any website.

A couple of issues I can think of with that decentralized approach:

* copyright - fuck it, it's decentralized, it can index whole books, maybe partnering with Anna's Archive or LibGen. Maybe have a copyright-respecting database and another one that doesn't respect it if you foresee the man coming down on the project. Maybe the results from the DB that doesn't respect copyright is merged at query-time with the one that does. Or maybe, the DB that doesn't respect copyright is just a superset of the copyright-respecting DB. I don't know how easy it would be to simultaneously search more than 1 DB.

* privacy - it could run over Tor or at least allow people to access it via Tor. The privacy of the cryptocurrency also seems doable - we have Monero and other private coins but I'm not sure how easy it would be to implement private micropayments with these.

* spam, intentionally wrong archives/crawls - pay the people who submit sites something so they financial motivation to not lie. Some consensus-based reward mechanism could be used, not sure which one

* moderation, illegal content - we don't care about copyright but likely don't want real CSAM, real animal abuse and other obviously awful content. Rewards should also be able to be used somehow for moderators or for people flagging content. We might even have a decentralized way to flag/tag content for anything at all - "AI generated" or "human generated", "small web", "uses Cloudflare", etc..

* how the distributed database actually works, how searching it works, who connects to whom when making a query and so on. I hope there are smart people with knowledge on such systems (not me lol) who can shed some light on whether it's possible and how.

DuckDuckGo is about to surge!

If you really want the best search engine, ChatGPT with thinking mode enabled is by far the best search engine technology that exists today. There's nothing that comes close.

This one is also stupid:

> But if a search engine were to operate without ads, could it still make money?

> That’s what Kagi is trying to accomplish. For $5 per month — or $10 for unlimited searches — you can access an ad-free search engine without AI overviews.

UUuuh ChatGPT exists for $20 per month and does the best searches (amongst other things) and is also ad free.

----

Edit: getting downvoted

Firstly, it is pretty obvious to me and everyone else reading that this specific concern that the content producers won't make money is largely performative and insincere.

From the article:

> many users see this as yet another example of a tech company squeezing AI agents and chatbots into everything it can, making it impossible to navigate the internet without encountering a chatbot

This is purely ideological. I can say this because Ads, which are the very thing keeping content alive, is the very thing opposed generally by the same people.

Secondly, it is exactly Google, the company that pioneered ads, the thing that people take an issue with, are the ones doing this. Surely such a company knows how to balance ad revenue and long term user growth. If your concern is so valid that content creators won't make money, why do you think Google is doing all of this, especially when they are bound to lose their main source of revenue? It was Google that even made content creation possible by providing revenue.

Thirdly, and I can't prove it but I mean this in an normative and a positive way: AI for search is good for humanity, good for content creators as well. The large second order effects can't be explained but making it quick and easy for users to search and provide results for complicated prompts is a _good_ thing. I generally do click people's blogs and learn more about them and follow them.

In fact, if the concern were actually sincere, we would be seeing the second order effects more lucidly: lower SEO spam and higher quality publications.

I'm already seeing newer forms of content monetisation in the form of substack etc. This is by far a better, more aligned approach than SEO cat and mouse games. I also see advertisements working better because a rich prompt has better CTR which opens up a potentially better content economy. But I predict this very thing would infuriate the same people even more. "How dare I get more relevant ads and make Google richer??"

I'm sure what I typed up would be downvoted because of ideological reasons, but the few that think a bit more deeply might agree and see my point. Performative concern is tiring.