• Usenet-Rewind: search engine for Usenet messages back to 1981

    From Craig Stadler@3:633/10 to All on Sun Sep 6 06:12:18 2026
    I spent the last year gathering and normalizing as many resources for
    Usenet messages and built a search engine over public Usenet
    discussions going back to 1981. Posting here because this seemed like
    the one group that would actually care.

    https://www.usenet-rewind.com/

    Since Google Groups stopped indexing new Usenet content a
    while back, and search over its historical archive has always been
    rough- I wanted something that treats this as a real research
    archive -- full-text search, date-range/filtering,etc and most
    importantly as much Usenet data as possible going back as far as I could
    find.

    The index currently holds (to date) roughly 980 million messages and is
    still growing. Sources are archival backups, data donations, and ongoing
    crawls of several thousand still-active public NNTP servers. Binary and yEnc-encoded content is stripped where possible to keep the index
    focused on text.

    The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu
    22.04.5 LTS & custom Python scripts, all on nvme disk.

    This is public archival content, same legal basis as DejaNews and
    Google Groups before it. There's a removal process for anyone who
    wants their own posts taken down, and author contact info is masked
    by default.

    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Nuno Silva@3:633/10 to All on Sun Sep 6 10:01:27 2026
    On 2026-09-06, Craig Stadler wrote:

    I spent the last year gathering and normalizing as many resources for
    Usenet messages and built a search engine over public Usenet
    discussions going back to 1981. Posting here because this seemed like
    the one group that would actually care.

    https://www.usenet-rewind.com/

    Since Google Groups stopped indexing new Usenet content a
    while back, and search over its historical archive has always been
    rough- I wanted something that treats this as a real research
    archive -- full-text search, date-range/filtering,etc and most
    importantly as much Usenet data as possible going back as far as I could find.

    The index currently holds (to date) roughly 980 million messages and is
    still growing. Sources are archival backups, data donations, and ongoing crawls of several thousand still-active public NNTP servers. Binary and yEnc-encoded content is stripped where possible to keep the index
    focused on text.

    The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 22.04.5 LTS & custom Python scripts, all on nvme disk.

    This is public archival content, same legal basis as DejaNews and
    Google Groups before it. There's a removal process for anyone who
    wants their own posts taken down, and author contact info is masked
    by default.

    Can you consider setting a width and/or height for SVG icons for better compatibility? Or perhaps not use SVG icons at all?

    I can get a paginated list of newsgroups, but they're not linkified, how
    is this supposed to be used?

    --
    Nuno Silva

    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Peter Flass@3:633/10 to All on Sun Sep 6 08:02:37 2026
    Thank you! This is definitely a worthwhile service. I'll have to try it out.

    On 9/5/26 23:12, Craig Stadler wrote:
    I spent the last year gathering and normalizing as many resources for
    Usenet messages and built a search engine over public Usenet
    discussions going back to 1981. Posting here because this seemed like
    the one group that would actually care.

    https://www.usenet-rewind.com/

    Since Google Groups stopped indexing new Usenet content a
    while back, and search over its historical archive has always been
    rough- I wanted something that treats this as a real research
    archive -- full-text search, date-range/filtering,etc and most
    importantly as much Usenet data as possible going back as far as I could find.

    The index currently holds (to date) roughly 980 million messages and is
    still growing. Sources are archival backups, data donations, and ongoing crawls of several thousand still-active public NNTP servers. Binary and yEnc-encoded content is stripped where possible to keep the index
    focused on text.

    The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 22.04.5 LTS & custom Python scripts, all on nvme disk.

    This is public archival content, same legal basis as DejaNews and
    Google Groups before it. There's a removal process for anyone who
    wants their own posts taken down, and author contact info is masked
    by default.


    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Peter Flass@3:633/10 to All on Sun Sep 6 08:12:25 2026
    On 9/5/26 23:12, Craig Stadler wrote:
    I spent the last year gathering and normalizing as many resources for
    Usenet messages and built a search engine over public Usenet
    discussions going back to 1981. Posting here because this seemed like
    the one group that would actually care.

    https://www.usenet-rewind.com/

    Since Google Groups stopped indexing new Usenet content a
    while back, and search over its historical archive has always been
    rough- I wanted something that treats this as a real research
    archive -- full-text search, date-range/filtering,etc and most
    importantly as much Usenet data as possible going back as far as I could find.

    The index currently holds (to date) roughly 980 million messages and is
    still growing. Sources are archival backups, data donations, and ongoing crawls of several thousand still-active public NNTP servers. Binary and yEnc-encoded content is stripped where possible to keep the index
    focused on text.

    The index was built using Apache Solr 10, MariaDB 12 (rocksdb), Ubuntu 22.04.5 LTS & custom Python scripts, all on nvme disk.

    This is public archival content, same legal basis as DejaNews and
    Google Groups before it. There's a removal process for anyone who
    wants their own posts taken down, and author contact info is masked
    by default.

    Too bad there isn't some way to get rid of the porn. Search for "cp-v"
    (with the quotes) retrieves the following as the first hit:
    VEB ** Submissive Daddy's Girl Whitney Does It All ** CP@V+
    or else we need to know the rules for searches.

    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Craig Stadler@3:633/10 to All on Sun Sep 6 16:55:47 2026
    Nuno Silva <nunojsilva@invalid.invalid> wrote in news:117ja57$26mtg$1@dont- email.me:

    Can you consider setting a width and/or height for SVG icons for better compatibility? Or perhaps not use SVG icons at all?

    I can get a paginated list of newsgroups, but they're not linkified, how
    is this supposed to be used?


    I fixed the SVG size issue (I think!) and also newsgroups are now clickable but it shows ALL messages inthat group sorted by oldest first. Let me know
    if you had something else in mind, thanks for the feedback.

    -Craig

    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Nuno Silva@3:633/10 to All on Mon Sep 7 10:51:55 2026
    On 2026-09-06, Craig Stadler wrote:

    Nuno Silva <nunojsilva@invalid.invalid> wrote in news:117ja57$26mtg$1@dont- email.me:

    Can you consider setting a width and/or height for SVG icons for better
    compatibility? Or perhaps not use SVG icons at all?

    I can get a paginated list of newsgroups, but they're not linkified, how
    is this supposed to be used?


    I fixed the SVG size issue (I think!) and also newsgroups are now clickable but it shows ALL messages inthat group sorted by oldest first. Let me know if you had something else in mind, thanks for the feedback.

    Thanks, a (very) quick glance showed it is already much better than it
    was on browsers which have the SVG size issue, and groups can be clicked
    on! :-)

    I'll try to have a longer glance at it when time allows.

    --
    Nuno Silva

    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Mark Shaw@3:633/10 to All on Thu Sep 10 00:52:58 2026
    Craig Stadler <cstadler18@hotmail.com> wrote:

    I spent the last year gathering and normalizing as many resources for
    Usenet messages and built a search engine over public Usenet
    discussions going back to 1981. Posting here because this seemed like
    the one group that would actually care.

    https://www.usenet-rewind.com/

    Thank you for this. I had a look at my old stomping grounds at rec.pets.dogs.behavior, and the first thing that came up in my
    search was a response to someone who got taken in by the resident
    newsgroup whackjob.

    Good times, good times.

    --
    Mark Shaw moc TOD liamg TA wahsnm ======================================================================== "Ironic that the two O's in 'cooperate' insisted on
    having their own separate sounds...." -Nathan W. Pyle

    --- PyGate Linux v1.5.19
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)