My Pain Points writing a Telegram Bot (in Python)

Most of my friends who use Telegram don’t use bots. Why?

I tried to write a Telegram Bot in Python this weekend, and there was a lot of friction. Let me outline my pain points. I’d love to be proven wrong if I’m mistaken about any of these points! Here’s what sucked for me:

All good bot names are squatted.
Painful to measure and improve engagement
Tricky to have a bot with memory

I suspect many other Telegram bot writers are having these issues. But I really like the potential of the Telegram bot ecosystem. Let’s dissect these issues one-by-one.

All good bot names are squatted.

The onboarding process just leaves you feeling humiliated. Here’s why.

First thing you have to do to write a Telegram Bot is message the BotFather. Cool, fair enough. You just need a bot name to get started, and it must end in “bot”. Simple, yeah?

Really? Pay you for the bot username that Telegram gave you for free?

Well it turns out that your preferred bot name is taken, and inactive. You’ll come up with a second-best bot name. Taken and inactive. In fact, it’s incredibly difficult to find a single good bot names that is available. Most English words, including arcane ones, are gone. Some people are actively squatting bot usernames for sale, and aren’t even hiding how sleazy they are about this.

There’s a sorta vague policy from Telegram about how to get a squatted handle, and how long it will take:

“Due to the fact that one account can register multiple bot and channel usernames, we reserve the right to recall usernames assigned to unused bots and channels, as well as openly squatted usernames.

Please contact our in-app Support if you have more questions (Settings — Ask a question). Notice that we rely on volunteers support in Telegram (so please be nice). Ping us on Twitter if you get no response within a day or so.”

The BotFather seems to encourage you to message him if you build an active bot. The whole onboarding process leaves you feeling disheartened and lacking confidence that you can claim the bot handle you really want, if you invest the time to build a good bot.

Hard to save bot state

So you’re ready to code. The API is rich and python-telegram-bot wrapper is an active project with a strong community and good docs. You want to be up-and-running fast, so you don’t want to do devops or maintain a server. I personally use Heroku because I’m familiar with it, and something serverless like AWS Lambda or the Google Cloud Functions is even cooler.

I want to log my bot’s JSON inputs and outputs to Mongo, because:

Structured logs let me write analytics dashboards, so I can measure and improve my bot’s engagement.
I’d like to have a bot with memory, so it can have rich interactions with its users over time.

I’m just going to log the raw JSON of everything, and later on I’ll transform it into a collection with a saner schema and some indexes.

Turns out, serializing your bot’s inputs and outputs to JSON will take you more than five minutes. It might actually take you an hour or even longer to get it right. (Please! I’d love to be proven wrong.)

You want to serialize your bot’s I/O and…

The python-telegram-bot debug logging contains nice sane JSON in stdout, but I couldn’t figure out a simple way to instrument a log-sink to a structured database. I could try archiving the stdout text and parsing out the JSON, but that’s ugly.

What I ended up doing is every time I receive a bot input, I take the input (usually an Update python class object) and serialize it JSON and the store it in Mongo. And after I send a message, I also do that. So my code is littered with a lot of boilerplate. Even this was tricky:

I don’t want to write custom JSON serialization for each Telegram class I use, because I want to be up and running fast.
to_dict is not implemented consistently through the API, so I can’t rely upon that.

I ended up writing a little function to use jsonpickle to convert Python class objects to a JSON string and the converting the JSON string to a Python dict, so I could insert it into Mongo. Kinda janky, particularly because your object ends up littered with many null/None fields, but I don’t feel like recursing through a dict with arbitrary value types to clean this up myself. I think the right approach would be to go straight from Python class object to JSON-serializable dict (without using a JSON string in the middle), but vars() and __dict__ only converted objects in a shallow way, not recursing into Python objects that are values, so that caused JSON serialization errors.

There is something called de_json in python-telegram-bot, but even that is intentionally not implemented consistently through the API. In that ticket, one dev writes:

The only case where this would be bad is if someone is serializing telegram objects as json and then deserializing them afterwards, but I don’t believe that should be a usecase we should support since

1. We don’t use it in the library internally anywhere

2. Pickle exists

Personally, I dislike that argument. Discouraging people from serializing to and from JSON is harmful, because it makes it harder for bots to be stateful. And Pickle is a bad alternative because 1) you need a server 2) it’s not a database and 3) human-readable formats are better, all things being equal. (In grad school, my advisor taught me to take my cryptic binary formats and convert them to human-readable formats. He argued that human-readable formats get read by humans more, and thus lead to more debugged and better quality code.)

I mean, something must exist because the JSON serialization in the Telegram debug log is actually really nice. I just can’t figure out how to get that JSON into my Mongo easily.

There’s some new functionality to make your bot persistent. Here’s a short wiki about it. There is only one persistent bot example in python-telegram-bot codebase, and I have no idea if it pickles to the filesystem (won’t work for Heroku or serverless), or if it persists to Telegram’s servers or what.

I think if I want to do this correctly, I should be using the DictPersistence class, but I’ve found it hard to find simple examples and am still scratching my head over how it works. I’m reasonably good at Googling and skimming docs, so my point is that if this functionality exists it’s definitely not obvious.

What I really want is python-telegram-bot/examples to have a simple bot example, where every bot input and output that every conceivably occurs will be stored to a database. Or, at least, an example where every input and output dict object is available in some callback, so I can quickly tweak that code.

Why this matters

The impact is:

There’s a lot of friction if you want to measure and improve your bot’s engagement with users.
There’s a lot of friction if you want your bot to have rich conversations with users, that includes memory.

I suspect this is why a lot of long-tail Telegram bots don’t have good engagement, and are stateless. Which is a pity. I think the community would benefit greatly if serializing all bot I/O to JSON or a JSON-db like Mongo were obvious native functionality.

Conclusions

Telegram bots have wonderful potential. They expose a lot of cool functionality, and the official python-telegram-bot is good. But the pain points that I experienced will also hold back other members of the Telegram bot writing community. That leads to a less satisfying bot experience for everyone, especially users.