Hello All.

I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

6/24 05:36:27p 3008 HTTP [20.171.207.62] Request 1825: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_ads&thread=105 HTTP/1.1
6/24 05:36:27p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
6/24 05:36:28p 3008 HTTP [20.171.207.62] Request 1826: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=177 HTTP/1.1
6/24 05:36:28p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
6/24 05:36:29p 3008 HTTP [20.171.207.62] Request 1827: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=801 HTTP/1.1

Is there a way to stop someone from using a BOT to take the files besides turning off the server?


RickV

--- SBBSecho 3.28-Win32
* Origin: Live Wire BBS [livewirebbs.com:1025] (1:154/110)
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: Web server and BOT's
By: Richard Vonzel to All on Wed Jun 25 2025 04:25 am

I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

6/24 05:36:27p 3008 HTTP [20.171.207.62] Request 1825: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_ads&thread=105 HTTP/1.1
6/24 05:36:27p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
6/24 05:36:28p 3008 HTTP [20.171.207.62] Request 1826: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=177 HTTP/1.1
6/24 05:36:28p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0

Is there a way to stop someone from using a BOT to take the files besides turning off the server?

yeah there's a lot of ways. you can also just make it so logged in users have access to your msgs and files. please checkout the wiki and just read through it in your free time.

The synchronet docs overall are a good source of information, unlike most other bbs softwares.

you're running a service available to the entire internet. someone could go in and do some horrible stuff if you don't secure your system.
---
þ Synchronet þ ::: BBSES.info - free BBS services :::

Richard Vonzel wrote to All <=-

Is there a way to stop someone from using a BOT to take the files
besides turning off the server?

Make all of your file content available only to authenticated users. I
think my guest user has an access string of LEVEL 40, so I made all of
my echo content LEVEL 50. If a guest hits the file areas, they won't see anything. If an authenticated user hits them, they'll see the areas.

You could do something similar with the file echoes. Mine are wide open,
I don't have anything I need to limit access to - although the bandwidth
would be a pain to lose to bots.



--- MultiMail/Win v0.52
þ Synchronet þ .: realitycheckbbs.org :: scientia potentia est :.

Re: Web server and BOT's
By: Richard Vonzel to All on Wed Jun 25 2025 04:25 am

Hello All.

I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

6/24 05:36:27p 3008 HTTP [20.171.207.62] Request 1825: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_ads&thread=105 HTTP/1.1
6/24 05:36:27p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
6/24 05:36:28p 3008 HTTP [20.171.207.62] Request 1826: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=177 HTTP/1.1
6/24 05:36:28p 3008 HTTP [20.171.207.62] User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)
6/24 05:36:29p 3008 HTTP [20.171.207.62] Request 1827: GET /?page=001-forum.ssjs&sub=fsxnet_fsx_gen&thread=801 HTTP/1.1

Is there a way to stop someone from using a BOT to take the files besides turning off the server?

You can disable the Files tab in ecWebv4 if I recall correcctly.
--
digital man (rob)

Breaking Bad quote #16:
Thinking Operation Breath Mint evertime you and me are on a stakeout together. Norco, CA WX: 83.1øF, 46.0% humidity, 15 mph W wind, 0.00 inches rain/24hrs
---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: Web server and BOT's
By: Richard Vonzel to All on Wed Jun 25 2025 04:25:24

I turned on the Synchronet Web server, works great, hsad my website running. Had my BBS's files available for access thru it. Yesterday I guess a BOT got on my site and jst downloaded all the file I had available. Here's the log entry for this,

I have found gptbot ignores a global disallow rule in robots.txt

User-agent: *
Disallow: /

But you could try adding

User-agent: gptbot
Disallow :/

It could take a few days but you'll see when they start to pickup your robots.txt file.

Another option would be to just block the ip ranges altogether.

Another option would be to block the published subnets. I have these blocked using ipsets

4.227.36.0/25
20.171.207.0/24
52.230.152.0/24
172.182.204.0/24
20.171.206.0/24
172.182.214.0/24
20.125.66.80/28
172.182.215.0/24

That seems to be the easiest way and just don't deal with them at all.

---
þ Synchronet þ End Of The Line BBS - endofthelinebbs.com

not able to up/download QWK message packet
*?

I did receive a Fidonews file..

---
ï¿­ Synchronet ï¿­ Vertrauen ï¿­ Home of Synchronet ï¿­ [vert/cvs/bbs].synchro.net

Re: Web server and BOT's
By: Mrsteve to nelgin on Thu Jun 26 2025 09:51:55

not able to up/download QWK message packet
*?

I did receive a Fidonews file..

That's nice but what does it have to do with web server bots?

---
þ Synchronet þ End Of The Line BBS - endofthelinebbs.com