Well, with the latest news I'm contemplating now, whether it makes sense to continue any development or just wait for the clarity. 🤦‍♂

I'm so happy I'm not working at Twitter. 😆

https://www.nytimes.com/2022/11/11/technology/elon-musk-twitter-takeover.html?unlocked_article_code=AAAAAAAAAAAAAAAACEIPuonUktbfqYhlSVUaAibJUNMnqBqCgvfeh7E4gXm7KTWOSnNe0OUJH4WavlvEZqdkedYpmTSHQdsLeJkeeMtP9M4NdUp8V1vv5ZKehJUOJyhypYzsD2o_gMOIV71i_Xq3NTXhbe4jnbT67krWYTa-W6bahiAlNx8l9cU-DSyi2XQOz6fBFeNzy4M-ia9nXsYmMG9GMCqavPDoCAF9OMGFbnzc6wo0W-JbWlLDjILWquJAIEgJVwWwHD4o6n086dhcJNsUIK34ShUic8P9irwQYXd6PIfESts0aVSRnu2b8wj7vw&referringSource=articleShare

The bot generally sends photos and videos by passing URLs to Telegram. Telegram then itself retrieves photos and videos. But sometimes Telegram fails, and I don't know why. Examples are here and here.

Now, in this case the bot tries to actually download failed files and upload them to the Telegram. In the sandbox, it worked with the tweets above. Let's see whether it will work in the real life.

Just to make these cases visible, bot posts these media with "(@)" prefix, not the usual "(#)". Here is an example.

There are still scenarios when this will fail. Twitter allows uploading videos up to 250Mb, and Telegram only up to 50Mb. But I will be solving this issue only when it starts appearing on the regular basis.

There was a duplicate (already removed) of this post. I've started investigating this while having M113A3 incident flashbacks.

Twitter API had connection time outs for about an hour, thus resulting in Lambda timeout. The Lambda timeout ment that bot didn't save the posted tweet, so it posted it again.

Looks like an occasional hiccup. I still need to think, whether it make sense to save the last posted tweet as soon as it's posted.

I realized that there were no post in @UAWeapons since 1 Feb 2023 to 8 Feb 2023.

Short investigation revealed that the bot failed posting this tweet.

Quick fix: marking this tweet as posted, so it's skipped.

Long term fix needs proper investigation.

About a week ago, Twitter announced that they will stop the free access to their API on 9 Feb 2023.

There is still no further information (how much? for what?), so I'm waiting for details.

If the cost is prohibitive, I will be checking other options to access the data. But this will take some time.

It took me almost two weeks of slacking and an issue appearing in another channel, @OSINTtechnical_twitter, to fix it.

The root cause was in Google Maps URLs containing a single quote character.

Now fixed and running:
https://news.1rj.ru/str/OSINTtechnical_twitter/2138

If video was sent by uploading (this happens for bigger video files, and you can see it by "(@)" prefix instead of "(#)"), Telegram ignored its dimensions and display it as a square: https://news.1rj.ru/str/UAWarReports/9802

In order to see it with right dimensions, you had to download it and use external player.

Now, I have added passing width and height information, so these videos shall be displayed properly.

The first video sent with correct dimensions: https://news.1rj.ru/str/oryxspioenkop_tw/6033

📢 The free access to Twitter API will end on 1 May. 📢

This channel uses Twitter API to retrieve tweets for posting, and I can't afford paying $100 every month for a pet project. But I don't want to drop it either.

So, with the help of @nakedbotch, the author of @getnaked_bot and @IGSpyBot (highly recommend to check out both of them) I am trying to look for a workaround.

There is a good chance that every now and then, some things will be broken. There is also a chance, that at some point, the project will stop working completely. Let's see - I'll try to keep it alive as long as I can - with a meaningful effort.

Please fasten your seatbelts and expect some turbulence coming!

Well, just in a 2-3 days, the main share of the work is done.

Featurewise, it should be matching the current set. There is a couple of missing things under the hood like getting numerical user ID via twitter name, but you should not see it.

I am sure some issues will surface as well as Twitter will be changing some items. I will keep an eye on them.

The first issue found: tweets which are marked as aged restricted are not displayed. 🙁

For example video and text from the source tweet here:
https://twitter.com/deaidua/status/1651850847351980033

Over last week, the bot got broken twice: first, it changed request parameters, and second, it started requiring guest cookie in a request.

It took me a day or two to notice those issues, but now I fixed it. I also plan to improve monitoring, so in case these issues persist, I will be notified.

Couple of posts (one in English, another in Russian) were too long to be posted as a single Telegram message. Telegram limits a single (non-media) message to 4 096 characters, and Twitter nowadays lets people post up to 10 000 characters long posts.

Incident fixed those channels by splitting and posting messages manually. This broke the clogging, and all other message are being posted by bot as I write this.

Short term fix planned: set the monitoring for this kind of issues, so if they occur, I will be notified and will do the manual fix.

Long term fix planned: come up with the message splitting mechanism.

Today, I woke up to my worst nightmare related to the bot: hundreds of duplicate posts in multiple channels. The reason was this tweet with huge video. Bot was failing to upload it to Telegram, and crushing without storing any information about previousely sent posts. So, next time it was resending posts and trying to upload video again...

Incident is fixed, duplicates mostly removed, proper fix planned for... I don't know when. 🙂

Kinda fixed. 🥴

📣 Well, it looks to be over. 📣

Tonight, without any prior announcements, Twitter has disabled any access to tweets withot login. This means that the bot can't retrieve the data anymore, so I am shutting it down.

I'll keep the channels, but there will be no updates unless Twitter reverses its decision or I will find a way to bypass this limitation (which is unlikely).

It's very ironic and sad to see how two big platforms, Twitter and Reddit, go down the closure pass simultaneousely.

Thank you for staying here for all this time.

I'm not going to spend my quota to verify, whether it's true or now. 😆