Thread

🛡️
may have spent the last 8 hours building a fulltext search engine in nostrdb. I made the index as space efficient as possible, they keys are stored in a compressed format and map words and word indices to note ids. So when you type “the quick brown fox” it will be able to return results with those exact words in sequence (or not if it can’t find a sequence). Testing it now 👀. Will release soon ™

Replies (28)

🛡️
Idea: limit the index to, say, the 64000 most used words including plurals and other variations), names (first and last), materials and brands. So keep typo’s and rare words out of the index, sanitizing the indrx makes its size much more manageable. </suggestion>
If you’re looking for a quick and dirty way to add fuzzy search and stemming, try tokenizing the lowercase string into character triplets, including spaces: [ th,e q,uic,k b,row,n f,ox ] and sorting by highest count of matching tokens.