[Feature]: Custom Sticker, Emoji & audio packs - your input needed

[Arturro43]

Feature

Custom Sticker, Emoji & audio packs

Custom Sticker, Emoji & audio packs can be really helpful for users, their experience and therefore they can help with SimpleX growth. Currently I know hundreds of people who can migrate from Discord, Telegram and other messaging platforms to more private alternative, but I cannot recommend using SimpleX right now, because I know how crucial custom stickers are for them.

Here is my idea of how Custom Resource Packs could work in SimpleX's decentralized environment.

Overview

Every user can create his own Resource Pack using bot or preferably in native (proposed) UI:

Note: Using "Resource packs" as a name for sticker, emoji or audio pack is not clear enough for (especially new) user. Users should know what's the meaning behind new settings page, when they see its contents it becomes pretty much self-explainatory.

Name "Resource Pack" comes from the fact you can create custom sticker, emoji and audio packs (more about them later). They are like additional "resources" to your SimpleX experience.

Custom emojis

When using text to communicate things like feelings can be lost. With custom emojis you can express emotions behind your text message in your own, unique way.

Custom stickers

You can also use stickers (known from messengers like for example Telegram) to express feelings, send your favourite meme or just sh*tpost with your friends.

Custom short audio files

Sometimes sending images is not enough. Have you ever been in a situation when your group chat is talking about something and a quote/sound effect would fit it perfectly? Me neither... Thanks to custom audio packs at least you are prepared;)

"Technical" overview

Resource Packs work just like sending image/audio to (group)chat, but it is parsed by client as "object" you can interact with (for example click to add a copy of Resource Pack to your SimpleX profile). Private Resource Packs cannot be added to "saved sticker (resource) packs" by anyone. Contents of private packs are also hidden.

When creating, Resource Pack is saved in user's local database. After finishing, user can decide if he wants to make his Resource Pack public or keep it private (hence name "0xA43 Personal Pack" in my UI project).

Resources used in pack needs to be identified in some way. Telegram is requiring user to assign emoji to a sticker. In my opinion, the best solution instead of assigning an emoji to a sticker would be to use some sort of identifier, for example $sticker_name$, this gives the author better way of defining what the sticker represents and potentially give it a unique name that can be easily remembered by sticker pack users who prefer to use "commands" instead of using the UI designed to select stickers. Unfortunately, this carries a theoretical possibility of deanonymization see possible privacy problems

Public Resource Packs

After creating new Resource Pack user can decide if he wants to make it public. After doing so, user assigns public name to his Resource Pack.

Resolving content from Public Resource Pack could work in few ways:

1. When sticker/emoji/audio is used on (group)chat for the first time, entire Resource Pack (except private packs) is copied to "database" or "cache" of that chat (which I assume every (group)chat user could have). Using item from Resource Pack is like sending request to this database to replace user's message with content from Resource Pack. (it could save bandwidth & storage)
2. After making them public, Public Resource Packs are stored on XFTP servers. We don't want XFTP servers to know anything about Resource Pack, especially their contents, so we need to use encryption. When publishing, user's client is generating two keys - let's call them base and share key. Base key is used when user sends sticker/emoji/audio - client is requesting content from XFTP server with content ID (more about them later). Client receives response containing encrypted file and by using base key client (locally) decrypts received content and sends message containing decrypted sticker/audio/emoji along with share key. Share key is used for two things - ~content discovery (when user wants to add Resource Packs to his saved packs it should be possible for user to view contents of said Resource Pack) and when user decides to add Resource Pack to his saved packs share key becomes a new base key for that user. To mitigate "origin-tracking" attack user's client is creating a slightly modified copy of saved Resource Pack - contents have different checksums to add layer of deniability - and sends his own version to XFTP server. This solution is not perfect, after some time, if there are many users of custom sticker packs, servers would require huge amounts of storage space just to store something completely non-critical.
3. We can use a mix of solution 1. and 2. - only users can store custom resource packs. Base and share key "architecture" would work the same, but content is requested only from custom pack users. It creates a new problem - what if custom pack user is offline or he deleted his profile? This problem could be resolved using "chat cache" - it could work only on group chats and if there's at least one user online and if he has cached entire Resource Pack.

As you can see, it is very difficult to come up with good idea of how custom stickers should work and we haven't even touched possible privacy problems custom stickers can cause... - that's why i need your input!

Using and saving Resource Pack

Resource Packs work just like sending image/audio to (group)chat, but it is parsed by client as "object" you can interact with (for example click to add a copy of Resource Pack to your SimpleX profile). Sharing resource packs is also very important. The way it looks in Telegram feels very native. To add a particular pack to the "vault" of your sticker/emoji packs, just tap on the sticker/emoji someone sent you and it opens the add screen, where you can also view the contents of the pack you want:

Private Resource Packs

If user decides to keep his resource pack private, content of his resource pack is not discoverable or saveable by others:

Possible privacy problems

Theoretically, using some sort of custom resource pack could contribute to the deanonymization of users.

1. "Origin-tracking attack" - as Evgeny said if somebody wants to track users, attacker can simply send slightly different ("poisoned") versions of the same pack to different users (for example with one additional sticker) and observe spread/exchange of "poisoned" resource packs between groups or in some (rare) cases even between individuals.
2. "Affiliation-tracking attack" - it's not a problem to have some sort of closed user community in which some member creates their own resource pack, and their members start using that resource pack. A potential observer could try to determine that a particular user is from a certain community (or at least potentially knows/uses a certain language) because they are using the same resource pack that the community in question was using (or the IDs in the resource pack are written in a particular language).

Possible mitigations

Users should know about risks associated with using custom resource packs regardless what other mitigations SimpleX could put in place, because some attacks are not preventable (see "proposed ui" above).

While "origin-tracking attack" is pretty much unpreventable if attacker created "poisoned" resource pack with for example sticker with added artifact only he knows about, there are some ways to minimize risk of deanonymization of users by implementing:

1. Mechanism that creates unique copies of resource packs for each and every user - when a resource pack is saved, a 1:1 copy of it is created (preferably in a slightly modified form so that, for example, the hash of a photo/audio files is differs from the original). Unfortunately, this brings up new issues:

• potential violation of copyrights (this can be resolved by informing the user who is creating the resource pack that by creating it, they are waiving their rights, as each person using their resource pack will have a unique (and slightly modified) copy of everything that is included in their resource pack)
• since each user owns a unique copy of the resource pack (preferably with the name changed; which may be the n-th copy of the copy), and someone (who loves resource packs) owns a huge amount of them then after some time it is hard to remember exactly what resources you already own. Lets say you liked a sticker/emoji/audio that you didn't even know you already owned - how are you supposed to know that you already own a particular resource pack? (This can be solved by: 1. preventing users from adding more resources to the resource pack 2. preventing users from editing the "IDs" assigned to the resource pack (essentially locking resource packs) in order to be able to 3. compare existing IDs of resources in the resource packs users already own. If specific IDs match (and, for example, there is a 100% similarity in the IDs occurring in one owned and a resource pack being added) then you can show the user prompt that he probably already owns a resource pack because it contains the same ID and indicate which one it is about and let user decide if he's sure he wants to add it to his "vault")

2. Requirement that each "ID" of the resource can contain only a-z characters, to minimize the risk of (still theoretical) deanonymization by assuming that a given user knows a certain language, because the ID of the resource contains special characters used only in a specific language. Unfortunately, this would lead to the introduction of a limitation of the ability to quickly choose a resource by using its ID for users who do not use languages based on Latin characters.

Conclusion

Custom Resource Packs can bring SimpleX user experience to a new level, but they pose risks that could potentially contribute to deanonymization of Custom Resource Packs users. We need to discuss the best approach to this problem together.

I can't wait to see your contributions :)

Thank you.

[nahuhh]

Can we add regular emojis first? lol

[Arturro43]

Can we add regular emojis first? lol

Of course! SimpleX Chat should focus on current problems/enhancements.

Implementation of custom stickers require A LOT of development and problem-solving. I've started this issue to discuss best approach to feature I think should exist in the future :)

[nahuhh]

I think the best ux is simple

1. Full emoji support for reactions
2. Full built in emoji keyboard
3. Ability to enter emojis by :laugh etc (follow element)
4. Tabs on emoji keyboard for emoji/gif/sticker*
5. Stickers are, imo, annoying and should be possible to disable them in your 1:1 or admin disable in group chats.

tldr: lets get going with full emoji keyboard + :emoji search + full emoji reactions = massive improvement to ux at very little dev cost (compared tp custom sticker packs)

blackberry wasted far too much time (and literally died before finishing..)
dont be blackberry.