• FTN Packet's message date question

    From slacker@21:3/193 to All on Tue Dec 10 14:49:01 2024
    Hi All,

    I'm trying to make sure that I'm importing messages with the right date and time based on the message date as well as the TZUTC kludge. What I noticed is that not all messages have a TZUTC kludge and the date of the message is given in their local time.

    I've also noticed that some messages have a TZUTC kludge but after adjusting the date of the message to UTC based on that, sometimes the date is still a little ahead of UTC. I'm guessing perhaps that one is just due to drifting origin BBS server times.

    Anyone have an idea how to parse the date when no timezone info is given? Am I thinking to much about this and should just be using the date when I import the message into my BBS instead of the actual message's date?

    Thanks!

    --- NE BBS v0.72 (linux; x64)
    * Origin: NE BBS - nebbs.servehttp.com:9223 (21:3/193)
  • From Digital Man@21:1/183 to slacker on Tue Dec 10 12:03:48 2024
    Re: FTN Packet's message date question
    By: slacker to All on Tue Dec 10 2024 02:49 pm

    Hi All,

    I'm trying to make sure that I'm importing messages with the right date and time based on the message date as well as the TZUTC kludge. What I noticed is that not all messages have a TZUTC kludge and the date of the message is given in their local time.

    That's right. It's dumb, but it's right. :-)

    I've also noticed that some messages have a TZUTC kludge but after adjusting the date of the message to UTC based on that, sometimes the date is still a little ahead of UTC. I'm guessing perhaps that one is just due to drifting origin BBS server times.

    The *date* is ahead, or the *time* is ahead? If the time is ahead, I'd say yeah, drift, but if your *hours* off (enough to put you into a future date), then I'd say eitehr the originating system is misconfigured or someone is modifying the date or TZUTC value of the message in-flight.

    Anyone have an idea how to parse the date when no timezone info is given? Am I thinking to much about this and should just be using the date when I import the message into my BBS instead of the actual message's date?

    When no timezone info is given, the best you can do is guess that it's either UTC or *your* local timezone. Unless you have some other way to know/guess the originator's timezone (nodelist maybe?).
    --
    digital man (rob)

    Rush quote #5:
    Some are born to rule the world, to live their fantasies
    Norco, CA WX: 70.0шF, 7.0% humidity, 7 mph E wind, 0.00 inches rain/24hrs
    --- SBBSecho 3.23-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (21:1/183)
  • From deon@21:2/116 to slacker on Wed Dec 11 08:29:51 2024
    Re: FTN Packet's message date question
    By: Digital Man to slacker on Tue Dec 10 2024 12:03 pm

    Howdy,

    When no timezone info is given, the best you can do is guess that it's either UTC or *your* local timezone. Unless you have some other way to know/guess the originator's timezone (nodelist maybe?).

    Depending on what you are doing mailer wize, during a BINKP (and EMSI) session, the remote's time is passed to you - from there you know their local time, and I think you'd know whether it was UTC or local.

    Example of what I see with clrghouz:

    :+ M_NUL [SYS SkyNet BBS] {"pid":6247}
    :+ M_NUL [LOC Medellin, Colombia] {"pid":6247}
    :+ M_NUL [ZYZ DavidG] {"pid":6247}
    :+ M_NUL [TIME Tue, 10 Dec 2024 07:54:14 -0500] {"pid":6247}
    :+ M_NUL [VER Mystic/1.12A48 binkp/1.0] {"pid":6247}
    :+ M_NUL [BUILD 2023/01/15 15:29:47 Windows/32] {"pid":6247}

    So you can see that David is UTC-5 and his localtime. It doesnt help if his packets have been sitting there a while (created hours/days ago) though - so you cant validate the time in the packet by recalculating it.

    It might be helpful nonetheless...


    ...лоеп
    --- SBBSecho 3.20-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From claw@21:1/210 to slacker on Wed Dec 11 07:14:52 2024
    On 10 Dec 2024, slacker said the following...
    Hi All,
    Anyone have an idea how to parse the date when no timezone info is
    given? Am I thinking to much about this and should just be using the
    date when I import the message into my BBS instead of the actual
    message's date?

    Thanks!


    File modification date. This would use the time it comes in to you instead. Might not be perfect but will semi solve the issue

    |23|04Dr|16|12Claw |14W0CLW
    |16|14Sysop |12Noverdu |14BBS |20|15Radio|10@|14HTTP://Noverdu.com:88
    |16|10 Standard ports for SSH/Telnet |04 WEB|14@|12HTTP://noverdu.com:808 |20|15Global Chat, Global Messaging and Games! |16|10Ditch the Unsocial Media

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: Noverdu BBS (21:1/210)
  • From slacker@21:3/193 to deon on Fri Dec 13 15:28:05 2024


    Depending on what you are doing mailer wize, during a BINKP (and EMSI) ses sion, the remote's time is passed to you - from there you know their local
    time, and I think you'd know whether it was UTC or local.

    In this case, I'm download the packets from clearing houz for the echomail areas.

    Here's an example from the Prison BBS ad from my logs:
    [2024-12-13 09:51:34.60784] [9230] [error] Parsed date time: 2024-12-14T03:01:22 is after current UTC date: 2024-12-13T14:51:34 even after TZ correction 0000 :: Using current UTC date for value.

    (Correction defaults to 0000 if kludge is missing)

    Here's the FTSC date from the packet:

    FTSC_DATE: 14 Dec 24 03:01:22

    So in this case, it was hours ahead. Clearing Houz is +11 hours. Does that effect any of the packet times that are sent to me?

    From Binkp logs:
    10:00 [9276] TIME Sat, 14 Dec 2024 02:00:02 +1100

    Thanks for the help!


    --- NE BBS v0.73 (linux; x64)
    * Origin: NE BBS - nebbs.servehttp.com:9223 (21:3/193)
  • From Exodus@21:1/144 to Slacker on Fri Dec 13 15:43:47 2024
    So in this case, it was hours ahead. Clearing Houz is +11 hours. Does that effect any of the packet times that are sent to me?

    Why would it? A packet is a packet and will be uncompressed and toss to the BBS. I don't get what the big problem is?

    ... Get the facts first - you can distort them later !

    --- Renegade v1.35/DOS
    * Origin: The Titantic BBS Telnet - ttb.rgbbs.info (21:1/144)
  • From deon@21:2/116 to slacker on Sat Dec 14 21:11:36 2024
    Re: Re: FTN Packet's message date question
    By: slacker to deon on Fri Dec 13 2024 03:28 pm

    Howdy,

    In this case, I'm download the packets from clearing houz for the echomail areas.
    Here's the FTSC date from the packet:

    FTSC_DATE: 14 Dec 24 03:01:22
    So in this case, it was hours ahead. Clearing Houz is +11 hours. Does that effect any of the packet times that are sent to me?

    So I just had a look at my code.

    With clrghouz, the packet date, is the date of youngest (most recent) message in the packet, in UTC.

    So if you collect mail regularly, and only get 1 mail packet, then it'll be the UTC time of the most recent message.

    If the message doesnt have a TZUTC kludge, then that "utc time" will probably not be UTC, unless the sender sends mail in UTC (and I imagine most wouldnt).

    My rational for doing that, was to enable a faster way of rejecting mail that is old - if the packet date has the timestamp of the youngest message (and thus the messages in it are older than that).


    ...лоеп
    --- SBBSecho 3.20-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From deon@21:2/116 to slacker on Sat Dec 14 21:29:14 2024
    Re: Re: FTN Packet's message date question
    By: deon to slacker on Sat Dec 14 2024 09:11 pm

    Howdy,

    FTSC_DATE: 14 Dec 24 03:01:22
    So in this case, it was hours ahead. Clearing Houz is +11 hours. Does that effect any of the packet times that are sent to me?

    One of the things I can do - which I've toyed with in the past, is to give packets (and possibly messages headers) in your local time. IE: When we connect, and I get a NUL Date message, with timezone information, by definition I know what timezone you are in and as I generate packets I can use that timezone for the packet header and message headers.

    I've not noticed that dates/times are a real problem though (we all know its not reliable), so I've not really bothered.

    And not all packet times use dates though (2.2) - and I use 2.2 with my BBS because its a 5d packet.


    ...лоеп
    --- SBBSecho 3.20-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From Digital Man@21:1/183 to deon on Sat Dec 14 16:33:33 2024
    Re: Re: FTN Packet's message date question
    By: deon to slacker on Sat Dec 14 2024 09:11 pm

    With clrghouz, the packet date, is the date of youngest (most recent) message in the packet, in UTC.

    Do you support (generate or parse) type 2.2 packets? I ask because type 2.2 packet headers don't include a date.
    --
    digital man (rob)

    Steven Wright quote #18:
    Hard work pays off in the future; laziness pays off now.
    Norco, CA WX: 61.2шF, 52.0% humidity, 2 mph SSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.23-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (21:1/183)
  • From deon@21:2/116 to Digital Man on Sun Dec 15 13:25:27 2024
    Re: Re: FTN Packet's message date question
    By: Digital Man to deon on Sat Dec 14 2024 04:33 pm

    Howdy,

    With clrghouz, the packet date, is the date of youngest (most recent) message in the packet, in UTC.

    Do you support (generate or parse) type 2.2 packets? I ask because type 2.2 packet headers don't include a date.

    Yes I do (both - generate and parse) - and I am aware of that.


    ...лоеп
    --- SBBSecho 3.20-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From slacker@21:3/193 to deon on Mon Dec 23 19:23:37 2024
    Thanks for all the info! Sorry for the late reply on this...

    I *think* I found the issue at least on some messages. It seems like it might be related to daylight savings and the kludge not getting updated on remote systems.

    Example this message from earlier today:
    AREA: FSX_STA :: MSG_ID: 21:1/100 67694287 :: FROM: 21:1/100 :: TO: 21:3/193 :: SUBJ: [NET1] Node Queue

    FTSC_DATE: 23 Dec 24 23:59:18 :: TZOFFSET: 1200

    BBS Log:
    [2024-12-23 06:04:11.85648] [13813] [error] Parsed date time: 2024-12-23T11:59:18 is after current UTC date: 2024-12-23T11:04:11 even after TZ correction 1200 :: Using current UTC date for value.

    I think coming from hub 1, the TZUTC offset should be 1300 now instead of 1200 for daylight savings, right? If I corrected for 1300, the time would have been valid and just 5 min before the current UTC time when I processed the packet.

    Link to the message on clrghouz: https://clrghouz.bbs.dege.au/echomail/view/635493

    I think I might be driving myself nuts for no reason lol. I poll every 30 min so current UTC of processing for received messages should be 'close enough'. I'll also take a look at using the packet date as you mentioned in a previous message. Maybe that would be a better choice than what I'm doing.. but again, I think I might just be over thinking this. :)


    --- NE BBS v0.76 (linux; x64)
    * Origin: NE BBS - nebbs.servehttp.com:9223 (21:3/193)
  • From deon@21:2/116 to slacker on Tue Dec 24 10:02:12 2024
    Re: Re: FTN Packet's message date question
    By: slacker to deon on Mon Dec 23 2024 07:23 pm

    Howdy,

    FTSC_DATE: 23 Dec 24 23:59:18 :: TZOFFSET: 1200

    BBS Log:
    [2024-12-23 06:04:11.85648] [13813] [error] Parsed date time: 2024-12-23T11:59:18 is after current UTC date: 2024-12-23T11:04:11 even after TZ correction 1200 :: Using current UTC date for value.

    I think coming from hub 1, the TZUTC offset should be 1300 now instead of 1200 for daylight savings, right? If I corrected for 1300, the time would have been valid and just 5 min before the current UTC time when I processed the packet.

    Yes, you are correct. I just looked at the packet I received from Hub 1,and it does have TZUTC 1200. It should be TZUTC 1300 to be correct, but since its generated from a script Paul might have 1200 hard coded.

    I'll also take a look at using the packet date as you mentioned in
    a previous message. Maybe that would be a better choice than what I'm doing.. but again, I think I might just be over thinking this. :)

    I dont recall what you are doing - but keep in mind, that the packet date I generate will be the time of the youngest message in the packet, in the time the source gave it to me (if the packet format has timestamps - from memory its only 2.2 that doesnt).


    ...лоеп
    --- SBBSecho 3.23-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From Avon@21:1/101 to deon on Fri Dec 27 14:01:41 2024
    On 14 Dec 2024 at 09:29p, deon pondered and said...

    One of the things I can do - which I've toyed with in the past, is to
    give packets (and possibly messages headers) in your local time. IE:
    When we connect, and I get a NUL Date message, with timezone
    information, by definition I know what timezone you are in and as I generate packets I can use that timezone for the packet header and
    message headers.

    does that mean I don't get to live in the future ...erm in the future :)

    Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

    --- Mystic BBS v1.12 A48 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From Avon@21:1/101 to slacker on Fri Dec 27 14:03:42 2024
    On 23 Dec 2024 at 07:23p, slacker pondered and said...

    I think coming from hub 1, the TZUTC offset should be 1300 now instead
    of 1200 for daylight savings, right? If I corrected for 1300, the time would have been valid and just 5 min before the current UTC time when I processed the packet.

    you're correct 1/100 UTC should be +1300 at the moment, apologies for this if it's not.

    Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

    --- Mystic BBS v1.12 A48 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From Avon@21:1/101 to deon on Fri Dec 27 14:06:17 2024
    On 24 Dec 2024 at 10:02a, deon pondered and said...

    Yes, you are correct. I just looked at the packet I received from Hub 1,and it does have TZUTC 1200. It should be TZUTC 1300 to be correct,
    but since its generated from a script Paul might have 1200 hard coded.

    My bad this is exactly what is happening... Humm... how to best fix? I can manually edit this to fix for now... but it's a pain to have to change twice yearly... Oh well :)

    Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

    --- Mystic BBS v1.12 A48 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From Avon@21:1/101 to Deon on Fri Dec 27 14:07:03 2024
    On 27 Dec 2024 at 02:06p, Avon pondered and said...

    My bad this is exactly what is happening... Humm... how to best fix? I
    can manually edit this to fix for now... but it's a pain to have to

    ..and it's done, the next bunch of reports should show +1300

    Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

    --- Mystic BBS v1.12 A48 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From deon@21:2/116 to Avon on Fri Dec 27 21:36:41 2024
    Re: Re: FTN Packet's message date question
    By: Avon to deon on Fri Dec 27 2024 02:06 pm

    Howdy,

    Yes, you are correct. I just looked at the packet I received from Hub 1,and it does have TZUTC 1200. It should be TZUTC 1300 to be correct, but since its generated from a script Paul might have 1200 hard coded.

    My bad this is exactly what is happening... Humm... how to best fix? I can

    "date +%z" should fix it for you. :)



    ...лоеп
    --- SBBSecho 3.23-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From deon@21:2/116 to Avon on Fri Dec 27 22:14:23 2024
    Re: Re: FTN Packet's message date question
    By: deon to Avon on Fri Dec 27 2024 09:36 pm

    Howdy,

    "date +%z" should fix it for you. :)

    And in case you need it (cant remember if the + is allowed in the TZUTC kludge), "date +%z:cut -c2-" (where : is a pipe symbol).


    ...лоеп
    --- SBBSecho 3.23-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From slacker@21:3/193 to deon on Sat Dec 28 04:30:14 2024

    And in case you need it (cant remem
    ber if the + is allowed in the TZUT
    C kludge), "date +%z:cut -c2-" (whe
    re : is a pipe symbol).

    *Pushes up glasses and does best 'nerd' voice* lol

    According to fsp-1001.002, the '+' shouldn't be included but it notes that 'robust implementations should be prepared to find (and ignore) a plus if it exists.'

    Btw Avon, your changes look good on my end from what I can tell. I don't see any error messages for TZ 1200 anymore in my logs.



    --- NE BBS v0.76 (linux; x64)
    * Origin: NE BBS - nebbs.servehttp.com:9223 (21:3/193)