759 lines
74 KiB
Markdown
759 lines
74 KiB
Markdown
# Encrypted Chat — Project Context
|
|
|
|
End-to-end encrypted chat with forward secrecy (X3DH + Double Ratchet, Signal Protocol).
|
|
Server stores and relays opaque blobs — never sees plaintext. RSA retained for login only.
|
|
|
|
## Files
|
|
|
|
| File | Lines | Purpose |
|
|
|------|-------|---------|
|
|
| `schema.sql` | ~158 | MySQL schema (users, devices, signed_prekeys, one_time_prekeys, conversations, conversation_members, group_invitations, messages, message_recipients, group_sender_keys, message_reads, image_uploads, user_profiles) |
|
|
| `db.py` | ~1245 | MySQL CRUD — one connection per call, `dictionary=True` cursors, returns dicts. Includes profile CRUD, `get_user_contacts()`, `update_conversation_creator()`, `get_conversation()`. Phantom user CRUD + `upgrade_phantom_user()`. Invitation CRUD. Group avatar. Device CRUD. Per-device prekey/session management. |
|
|
| `server.py` | ~1986 | Asyncio TCP server, handler dispatch, rate limiting, real-time notifications via `connected_clients` dict. Profile + avatar handlers. Online/offline status push. Leave group, delete conversation, group invitations, group avatar handlers. Phantom user support. Graceful shutdown. 4 asyncio.Lock guards (H4 fix). Device registration + per-device key bundles + per-device notifications. SPK age reporting in `get_prekey_count`. |
|
|
| `protocol.py` | ~114 | Newline-delimited JSON protocol, `ProtocolReader`/`ProtocolWriter`, `encode_binary`/`decode_binary` (base64). Constants: `VERSION`, `MAX_MESSAGE_BYTES`, `MAX_IMAGE_BYTES`, `MAX_FILE_BYTES`, `IMAGE_CHUNK_SIZE`. |
|
|
| `crypto_utils.py` | ~812 | Ed25519, X25519, AES-256-GCM, HKDF, PBKDF2, X3DH, `DoubleRatchet` (with state snapshot/rollback), `SenderKeyState` (with state snapshot/rollback). RSA for login only. ECP1 password-based key encryption format (600k PBKDF2 iterations). |
|
|
| `chat_core.py` | ~2555 | `ChatClient` class — session management, X3DH/ratchet encryption, local key storage, reconnect, profiles, file sharing, leave group, delete conversation, invitations, group avatar. Multi-device: per-device sessions, device_id persistence, device bundle cache. SPK rotation (7-day) with grace period. Used by CLI + GUI |
|
|
| `client.py` | ~382 | Interactive CLI client |
|
|
| `gui_client.py` | ~2591 | PyQt6 GUI — `AsyncBridge` QThread bridges asyncio <-> Qt signals, `MainWindow`, `UserProfileDialog`, connection indicator + auto-reconnect, online status, file sharing, leave group, unread badges, circular avatars in conv list, online green dot overlay, group invitations UI, delete conversation, group avatar support |
|
|
|
|
## Architecture & Data Flow
|
|
|
|
### Encryption: X3DH + Double Ratchet (Signal Protocol)
|
|
|
|
**Keys per user:**
|
|
- **RSA-4096** — Login challenge-response only (server stores public key). Password-encrypted with ECP1 format (PBKDF2 600k iterations + AES-256-GCM).
|
|
- **Identity Key (IK)** — Ed25519 (signing) + converted to X25519 (for DH in X3DH). Password-encrypted with ECP1 format.
|
|
- **Signed Pre-Key (SPK)** — X25519, signed by IK, uploaded to server. **Rotates every 7 days** (M4). Previous SPK kept for grace period (in-flight X3DH).
|
|
- **One-Time Pre-Keys (OPK)** — X25519, consumed on X3DH initiation, auto-replenished when count < 20
|
|
|
|
**DM flow:**
|
|
1. Alice fetches Bob's per-device key bundles (IK, SPK per device, OPK per device) -> X3DH per device -> shared secret per device
|
|
2. Double Ratchet initialized from shared secret — one session per (user, device) pair
|
|
3. Each message: symmetric ratchet (HMAC chain) -> message key -> AES-256-GCM
|
|
4. Each reply direction change: DH ratchet (new X25519 keypair) -> new root + chain keys
|
|
5. Per-device ciphertext — each recipient device gets individually encrypted blob
|
|
6. Self-encrypted copy uses SELF_DEVICE_ID sentinel, readable by all own devices
|
|
|
|
**Group flow (Sender Keys):**
|
|
1. Each sender has own SenderKeyState per group
|
|
2. Sender key distributed to members via pairwise Double Ratchet (as control DM with `_sender_key` field)
|
|
3. Group messages: symmetric ratchet on sender key -> AES-256-GCM
|
|
4. Same ciphertext replicated to all recipients (efficient)
|
|
|
|
### Protocol
|
|
|
|
Newline-delimited JSON over TCP (optional TLS). Fields: `type`, `status`, `data`, `request_id`.
|
|
Binary data encoded as base64 via `encode_binary()`/`decode_binary()`.
|
|
|
|
**Request/response pattern:** Client sends `{"type": "...", "request_id": "uuid", ...}`, server responds with same `request_id`. Notifications (push) have no `request_id`.
|
|
|
|
### Server notifications (push to connected clients)
|
|
- `new_message` — per-recipient ciphertext included
|
|
- `messages_read` — conversation_id + user_id + message_ids
|
|
- `message_deleted` — message_id + conversation_id
|
|
- `conversation_created` — conversation_id, name, created_by, members[] (pushed to added members)
|
|
- `member_added` — conversation_id, user_id, username, email (pushed to all members except requester)
|
|
- `member_removed` — conversation_id, user_id (pushed to removed member + remaining members)
|
|
- `group_invitation` — conversation_id, conversation_name, invited_by, invited_by_username (pushed to invited user)
|
|
- `conversation_renamed` — conversation_id, name, renamed_by (pushed to all members except renamer)
|
|
- `session_reset` — from_user_id, from_device_id (pushed to peer when session reset requested)
|
|
- `user_online` — user_id (pushed to contacts when user connects)
|
|
- `user_offline` — user_id (pushed to contacts when user's last connection drops)
|
|
- `online_users` — user_ids[] (sent to user on login — list of currently online contacts)
|
|
|
|
## DB Schema (schema.sql)
|
|
|
|
```
|
|
users: id, username, email (UNIQUE), rsa_public_key (TEXT), identity_key (BLOB 32B Ed25519), created_at
|
|
devices: id, user_id FK, device_name (nullable), created_at, last_seen_at
|
|
signed_prekeys: id, user_id FK, device_id (nullable), public_key (BLOB 32B), signature (BLOB 64B), created_at
|
|
one_time_prekeys: id, user_id FK, device_id (nullable), public_key (BLOB 32B)
|
|
conversations: id, created_at, name (nullable), created_by (nullable), avatar_file (nullable)
|
|
conversation_members: conversation_id + user_id (composite PK), joined_at
|
|
group_invitations: id, conversation_id FK, user_id FK, invited_by FK, created_at, UNIQUE(conversation_id, user_id)
|
|
messages: id, conversation_id FK, sender_id FK, sender_device_id (nullable), ratchet_header (BLOB JSON),
|
|
x3dh_header (BLOB JSON nullable), sender_chain_id (BLOB nullable), sender_chain_n (INT nullable),
|
|
created_at, deleted_at, image_file_id
|
|
message_recipients: message_id + user_id + device_id (composite PK), encrypted_content (BLOB), nonce (BLOB),
|
|
ratchet_header (BLOB nullable), x3dh_header (BLOB nullable)
|
|
group_sender_keys: conversation_id + sender_id + device_id (composite PK), chain_id (BLOB 32B), created_at
|
|
message_reads: message_id + user_id (composite PK), read_at
|
|
image_uploads: file_id (PK), conversation_id FK, uploader_id FK, file_size, completed, created_at
|
|
user_profiles: user_id (PK FK), phone, phone_visible, email_visible, location, location_visible, avatar_file, updated_at
|
|
```
|
|
|
|
Constant: `SELF_DEVICE_ID = "00000000-0000-0000-0000-000000000000"` — sentinel for self-encrypted copies and legacy rows.
|
|
|
|
Index: `messages(conversation_id, created_at)` for query performance.
|
|
|
|
## Server Protocol — All Message Types
|
|
|
|
### Pre-login (no session required)
|
|
| Type | Handler | Purpose |
|
|
|------|---------|---------|
|
|
| `register` | `handle_register_start` | Start registration (username, email, public_key, identity_key) |
|
|
| `register_confirm` | `handle_register_confirm` | Confirm with 6-digit code |
|
|
| `login_start` | `handle_login_start` | Get RSA challenge |
|
|
| `login_finish` | `handle_login_finish` | Respond with RSA signature -> session. Client sends `client_version`, server returns `server_version` in response. Also sends `online_users` and `user_online` notifications. |
|
|
| `get_user_info` | `handle_get_user_info` | Get user info + identity_key (by email or user_id) |
|
|
| `pairing_start` | `handle_pairing_start` | New device starts pairing (gets 8-digit code) |
|
|
| `pairing_poll` | `handle_pairing_poll` | New device polls for key payload |
|
|
|
|
### Post-login (session required)
|
|
| Type | Handler | Purpose |
|
|
|------|---------|---------|
|
|
| `upload_prekeys` | `handle_upload_prekeys` | Upload SPK + batch of OPKs (server verifies SPK signature) |
|
|
| `get_key_bundle` | `handle_get_key_bundle` | Fetch key bundle for X3DH (consumes one OPK) |
|
|
| `get_prekey_count` | `handle_get_prekey_count` | Check remaining OPK count + SPK age (`spk_created_at`) for rotation |
|
|
| `create_conversation` | `handle_create_conversation` | Create conversation — DMs auto-add both; groups add creator only + create invitations for others |
|
|
| `find_conversation` | `handle_find_conversation` | Find existing DM by email |
|
|
| `add_member` | `handle_add_member` | Create invitation for user to join group (was: direct add) |
|
|
| `remove_member` | `handle_remove_member` | Remove member (creator only) |
|
|
| `leave_group` | `handle_leave_group` | Voluntarily leave a group (transfers creator if needed, blocks DM leave) |
|
|
| `rename_conversation` | `handle_rename_conversation` | Rename group conversation (creator only, max 100 chars), pushes `conversation_renamed` to members |
|
|
| `delete_conversation` | `handle_delete_conversation` | Delete conversation — DMs: remove self; groups: creator-only, removes all members + files |
|
|
| `accept_invitation` | `handle_accept_invitation` | Accept pending group invitation → add to members, notify others |
|
|
| `decline_invitation` | `handle_decline_invitation` | Decline pending group invitation |
|
|
| `list_invitations` | `handle_list_invitations` | List user's pending invitations (with conv name + inviter username) |
|
|
| `list_conversations` | `handle_list_conversations` | List all user's conversations (includes avatar_file) |
|
|
| `send_message` | `handle_send_message` | Send encrypted message (ratchet_header + recipients[]) |
|
|
| `get_messages` | `handle_get_messages` | Get messages (returns per-user ciphertext, JOINs message_recipients) |
|
|
| `mark_read` | `handle_mark_read` | Mark messages as read |
|
|
| `delete_message` | `handle_delete_message` | Soft-delete message (sender only) |
|
|
| `rotate_keys` | `handle_rotate_keys` | Rotate RSA login key, disconnect other sessions |
|
|
| `pairing_claim` | `handle_pairing_claim` | Authorized device claims pairing code |
|
|
| `pairing_send` | `handle_pairing_send` | Authorized device sends encrypted key payload |
|
|
| `upload_image_start/chunk/end` | Image/file upload | Chunked encrypted upload (32KB chunks). `file_type` param: `"image"` (5MB limit) or `"file"` (50MB limit). |
|
|
| `download_image` | Image/file download | Chunked download with offset |
|
|
| `get_profile` | `handle_get_profile` | Get user profile (respects visibility for other users) |
|
|
| `update_profile` | `handle_update_profile` | Update own profile (phone, location, visibility toggles) |
|
|
| `update_avatar` | `handle_update_avatar` | Upload user avatar (base64, max 2MB, JPEG/PNG) |
|
|
| `get_avatar` | `handle_get_avatar` | Download user's avatar |
|
|
| `update_group_avatar` | `handle_update_group_avatar` | Upload group avatar (base64, max 2MB, JPEG/PNG, creator only) |
|
|
| `get_group_avatar` | `handle_get_group_avatar` | Download group avatar |
|
|
| `reencrypt_messages` | `handle_reencrypt_messages` | Batch re-encrypt message history with self-key (max 500/request, for device pairing) |
|
|
| `list_devices` | `handle_list_devices` | List all devices for current user |
|
|
| `remove_device` | `handle_remove_device` | Remove a device (not current device) |
|
|
| `session_reset` | `handle_session_reset` | Notify peer to reset corrupted Double Ratchet session (push `session_reset` to peer) |
|
|
|
|
## Key Classes & Functions
|
|
|
|
### crypto_utils.py
|
|
|
|
**Password-based key encryption (ECP1 format):**
|
|
- `PBKDF2_ITERATIONS = 600_000` — OWASP 2023 compliant
|
|
- `_encrypt_private_key(raw_bytes, password) -> bytes` — PBKDF2-HMAC-SHA256 + AES-256-GCM. Format: `_ECP1_MAGIC(4) + salt(16) + nonce(12) + ciphertext_with_tag`
|
|
- `_decrypt_private_key(data, password) -> bytes` — Detects ECP1 magic prefix, derives key, decrypts
|
|
|
|
**RSA (login only):** `generate_rsa_keypair()`, `serialize_private_key()` (ECP1 with password, PEM without), `serialize_public_key()`, `load_private_key()` (auto-detects ECP1 vs legacy PEM), `load_public_key()`, `rsa_sign()`, `rsa_verify()`
|
|
|
|
**AES-256-GCM:** `aes_encrypt(plaintext, key=None) -> (key, nonce, ct, tag)`, `aes_decrypt(key, nonce, ct, tag) -> plaintext`
|
|
|
|
**Ed25519:** `generate_identity_keypair()`, `serialize_ed25519_private()` (ECP1 with password, 32-byte raw without), `serialize_ed25519_private_raw()`, `serialize_ed25519_public()`, `load_ed25519_private()` (auto-detects ECP1 vs legacy PEM vs raw), `load_ed25519_public()`, `ed25519_sign()`, `ed25519_verify()`
|
|
|
|
**X25519:** `generate_x25519_keypair()`, `serialize_x25519_private()`, `serialize_x25519_public()`, `load_x25519_private()`, `load_x25519_public()`, `x25519_dh()`
|
|
|
|
**Key conversion:** `ed25519_private_to_x25519()` (SHA-512 + clamp), `ed25519_public_to_x25519()` (Montgomery u-coordinate)
|
|
|
|
**HKDF:** `hkdf_derive()`, `kdf_rk(root_key, dh_output) -> (new_root_key, chain_key)`, `kdf_ck(chain_key) -> (new_chain_key, message_key)`
|
|
|
|
**X3DH:** `generate_signed_prekey(identity_private) -> {private, public, signature, id}`, `generate_one_time_prekeys(count=50) -> [{private, public, id}]`, `x3dh_initiate(ik_private_ed, ik_public_remote_ed, spk_remote, spk_signature, opk_remote?) -> (shared_secret, ek_priv, ek_pub)`, `x3dh_respond(ik_private_ed, spk_private, ik_remote_ed, ek_remote, opk_private?) -> shared_secret`
|
|
|
|
**DoubleRatchet class:**
|
|
- `init_alice(shared_secret, bob_spk_pub)` — initiator, performs first DH ratchet
|
|
- `init_bob(shared_secret, spk_pair)` — responder, waits for first message
|
|
- `encrypt(plaintext) -> {header: {dh_pub, n, pn}, ciphertext, nonce}` — AAD = serialized header
|
|
- `decrypt(header_dict, ciphertext, nonce)` — handles DH ratchet step if new dh_pub, skipped messages. **State snapshot/rollback on failure (M9):** `_snapshot()` captures all mutable state before modifications, `_restore()` rolls back on any exception.
|
|
- `_snapshot() -> dict` / `_restore(snap)` — Snapshot: dh_pair, dh_remote, root_key, send/recv chain keys, counters, skipped dict. Used internally by `decrypt()`.
|
|
- `export_state() -> bytes` / `import_state(data) -> DoubleRatchet` — JSON serialization
|
|
|
|
**SenderKeyState class:**
|
|
- `__init__(sender_key=None)` — generates random 32B key if None
|
|
- `encrypt(plaintext) -> {chain_id, n, ciphertext, nonce}` — AAD = chain_id + message number
|
|
- `decrypt(chain_id_hex, n, ciphertext, nonce)` — fast-forwards chain if needed. **State snapshot/rollback on failure (M9):** snapshots chain_key, n, _known_keys before fast-forward, restores on exception.
|
|
- `export_key() -> bytes` — for distribution to group members
|
|
- `from_key(exported_key) -> SenderKeyState` — receiver initializes from exported key
|
|
- `export_state() / import_state()` — full state persistence
|
|
|
|
### chat_core.py
|
|
|
|
**Local key storage** (`~/.encrypted_chat/{email}/`):
|
|
```
|
|
private.pem / public.pem — RSA (login, ECP1 format when password-encrypted)
|
|
identity_private.bin / identity_public.bin — Ed25519 (ECP1 format when password-encrypted, 32B raw otherwise)
|
|
device_id.txt — This device's UUID
|
|
spk_private.bin / spk_id.txt — Current signed prekey
|
|
prev_spk_private.bin / prev_spk_id.txt — Previous SPK for grace period (M4, in-flight X3DH)
|
|
opk_private/{opk_id}.bin — One-time prekeys
|
|
sessions/{user_id}_{device_id}.bin — Double Ratchet states (per peer device)
|
|
sender_keys/{conv_id}.bin — Own sender key states
|
|
sender_keys_recv/{conv_id}_{sender_id}_{device_id}.bin — Received sender keys (per sender device)
|
|
```
|
|
|
|
Storage functions: `save_keys()`, `load_keys()`, `_save_identity_keys()`, `_load_identity_keys()`, `_save_spk()`, `_load_spk()`, `_save_prev_spk()`, `_load_prev_spk()`, `_save_opk_private()`, `_load_opk_private()`, `_delete_opk_private()`, `_save_session()`, `_load_session()`, `_save_sender_key_state()`, `_load_sender_key_state()`, `_save_recv_sender_key()`, `_load_recv_sender_key()`
|
|
|
|
**ChatClient attributes:**
|
|
- `private_key` / `public_key` — RSA (login)
|
|
- `identity_private` / `identity_public` — Ed25519
|
|
- `spk_private` / `spk_id` — Current SPK
|
|
- `_prev_spk_private` / `_prev_spk_id` — Previous SPK for grace period (M4)
|
|
- `opk_privates: dict[str, X25519PrivateKey]` — OPK private keys by ID
|
|
- `device_id: str | None` — this device's UUID (persisted to disk)
|
|
- `sessions: dict[str, DoubleRatchet]` — "user_id:device_id" -> ratchet (per peer device)
|
|
- `sender_key_states: dict[str, SenderKeyState]` — conv_id -> own sender key
|
|
- `recv_sender_keys: dict[str, SenderKeyState]` — "conv_id:sender_id:device_id" -> their key
|
|
- `_device_bundle_cache: dict[str, tuple[float, list]]` — user_id -> (timestamp, device_bundles) with 5-min TTL
|
|
- `_user_cache: dict[str, dict]` — user_id -> {identity_key, username, email}
|
|
- `connected: bool` — current connection state
|
|
|
|
**Key methods:**
|
|
- `register()` — Generates RSA + Ed25519, sends to server
|
|
- `confirm_registration()` — Confirms code, uploads prekeys (SPK + 50 OPKs)
|
|
- `login()` — Loads keys from disk (including prev_spk for grace period), RSA challenge-response, auto `_ensure_prekeys()`
|
|
- `_ensure_prekeys()` — Checks OPK count AND SPK age. Replenishes OPKs if < 20, **rotates SPK if >= 7 days old** (M4). Saves old SPK as grace period before generating new one.
|
|
- `_get_device_bundles(peer_user_id)` — Fetches per-device key bundles with 5-min TTL cache
|
|
- `_get_or_create_session(peer_user_id, peer_device_id, bundle)` — Loads from memory/disk or creates via X3DH, keyed by "user:device"
|
|
- `_process_x3dh_header(sender_id, x3dh_header, sender_device_id, spk_override?)` — Bob side of X3DH. `spk_override` param allows using previous SPK for grace period (M4).
|
|
- `send_message(conv_id, text, members, reply_to?)` — Routes to `_send_dm` or `_send_group_message`
|
|
- `_send_dm()` — Per-device Double Ratchet (encrypts for each of recipient's devices), self-encrypted copy with SELF_DEVICE_ID
|
|
- `_send_group_message()` — Sender Keys, distributes key if new (per-device)
|
|
- `_distribute_sender_key()` — Sends sender key as control message via per-device pairwise ratchet, includes sender_device_id
|
|
- `_decrypt_dm()` — Handles X3DH header for new sessions, returns None for control messages. On X3DH decrypt failure, retries with previous SPK (M4 grace period).
|
|
- `_decrypt_group()` — Uses received sender key chain
|
|
- `decrypt_notification()` — Returns None for control messages (sender key distribution)
|
|
- `get_messages()` — Batch decrypt, marks read, skips control messages
|
|
- `authorize_device()` — Exports RSA + Ed25519 only (simplified for multi-device — no session/sender key transfer)
|
|
- `pairing_wait()` — Imports RSA + identity key from paired device (new device generates own SPK + OPKs on login)
|
|
- `reconnect()` — Closes connection, re-establishes TCP + RSA login using in-memory keys
|
|
- `get_profile(user_id?)` — Gets user profile from server
|
|
- `update_profile(**fields)` — Updates own profile (phone, location, visibility)
|
|
- `update_avatar(image_data)` — Uploads avatar
|
|
- `get_avatar(user_id)` — Downloads avatar bytes
|
|
- `send_file(conv_id, file_path, members, reply_to?)` — Encrypt + chunked upload + send message with `file` payload
|
|
- `download_file(file_id, file_info)` — Chunked download + AES-GCM decrypt
|
|
- `leave_group(conv_id)` — Leave group, clean up local sender keys
|
|
- `rename_conversation(conv_id, name)` — Rename group (creator only)
|
|
- `delete_conversation(conv_id)` — Delete conversation, clean up local sender keys
|
|
- `accept_invitation(conv_id)` — Accept group invitation
|
|
- `decline_invitation(conv_id)` — Decline group invitation
|
|
- `list_invitations()` — Fetch pending invitations
|
|
- `update_group_avatar(conv_id, image_data)` — Upload group avatar
|
|
- `get_group_avatar(conv_id)` — Download group avatar
|
|
- `search_messages(conv_id, query)` — Search decrypted message cache (client-side only)
|
|
- `reset_session(peer_user_id, peer_device_id?)` — Delete local session + notify peer via server
|
|
- `handle_session_reset_notification(from_user_id, from_device_id?)` — Handle incoming session reset
|
|
|
|
### gui_client.py
|
|
|
|
**AsyncBridge (QThread):** Runs asyncio event loop, `schedule(coro)` queues coroutines, pyqtSignals emit results back to Qt main thread.
|
|
|
|
**Key signals:** `login_result`, `conversations_loaded`, `messages_loaded`, `message_sent`, `new_notification`, `messages_read_notification`, `message_deleted_notification`, `conversation_updated`, `connection_state_changed`, `profile_loaded`, `profile_updated`, `avatar_loaded`, `online_status_changed`, `online_users_loaded`, `file_sent`, `file_downloaded`, `group_left`, `conversation_deleted`, `invitations_loaded`, `invitation_result`, `invitation_received`, `group_avatar_loaded`, `group_avatar_updated`, `session_reset_notification`
|
|
|
|
**MainWindow:** Dark theme (Catppuccin Mocha), conversation list with circular avatars + online green dot overlay + unread count badges, message bubbles with colored left border, context menu (reply, delete, view image, download file), image thumbnails via QTextDocument resources (`thumb://{file_id}`), file cards with download links (`file://{file_id}`), connection indicator dot (green/red/orange), profile button, attach menu (Image/File), Leave Group button in group info, delete conversation button (trash icon in header), group avatar display + change in group info dialog, invitation list (amber border) above conversation list with right-click accept/decline.
|
|
|
|
**UserProfileDialog:** View (read-only) and edit (own profile) modes. Fields: avatar (circular crop), username, email, phone, location, visibility toggles. Avatar upload/download. Opened from "My Profile" button or user info button in group info dialog.
|
|
|
|
**Avatar system in conversation list:**
|
|
- `_avatar_cache: dict[str, QPixmap]` — user avatars by user_id
|
|
- `_group_avatar_cache: dict[str, QPixmap]` — group avatars by conv_id
|
|
- `_avatar_requested: set[str]` / `_group_avatar_requested: set[str]` — dedup download requests
|
|
- `_make_circular_avatar(pixmap, size=32)` — QPainter circular crop
|
|
- `_make_default_avatar(username, size=32)` — colored circle with initial letter (deterministic color from username hash)
|
|
- `_add_online_dot(avatar)` — green dot overlay bottom-right
|
|
- `_get_conv_avatar(conv)` — returns QIcon (DM: user avatar + online dot; group: group avatar or default)
|
|
- Periodic refresh every 2 minutes via `_refresh_timer` / `_on_periodic_refresh()`
|
|
|
|
## Important Implementation Details
|
|
|
|
### X3DH Header Caching
|
|
When `_get_or_create_session()` creates a new session via X3DH, it attaches the X3DH header as `ratchet._x3dh_header`. The next `_send_dm()` reads and deletes it. This ensures the X3DH header is only sent with the first message.
|
|
|
|
### Self-Encryption for DMs
|
|
Sender uses `derive_self_encryption_key(identity_private)` to encrypt their own copy of sent messages with a static AES key. Uses `SELF_DEVICE_ID` sentinel so all own devices can read it. This allows reading own sent messages when fetching history from any device.
|
|
|
|
### Sender Key Distribution as Control Messages
|
|
Sender keys are distributed via normal `send_message` protocol (per-device pairwise ratchet). The payload contains `_sender_key: {conv_id, key, sender_device_id}` field. On decryption, `_decrypt_dm()` detects this field, stores the sender key keyed by `"conv_id:sender_id:sender_device_id"`, and returns `None` (not shown to user).
|
|
|
|
### Group Messages: Dummy Ratchet Header
|
|
Group messages use `{"dh_pub": "00"*32, "n": 0, "pn": 0}` as ratchet_header because the server requires it, but groups use sender keys instead of Double Ratchet.
|
|
|
|
### Multi-Device Architecture
|
|
Each device has independent Double Ratchet sessions. Sessions are keyed by `"peer_user_id:peer_device_id"`. When sending a DM, the client fetches per-device key bundles via `_get_device_bundles()` and encrypts separately for each device. The server registers devices at login (`handle_login_finish`), assigns device IDs, and routes notifications with `device_entries` arrays (one entry per recipient device). Device IDs are persisted to `~/.encrypted_chat/{email}/device_id.txt`. Old session files (`{user_id}.bin`) are automatically migrated to `{user_id}_{device_id}.bin` on first load.
|
|
|
|
### Server Session Model
|
|
`connected_clients: dict[str, list[ProtocolWriter]]` — one user can have multiple connections (multi-device). `writer_device_map: dict[int, str]` maps `id(writer)` to `device_id`. Notifications are pushed to all connections except the sender's current one.
|
|
|
|
### Device Registration
|
|
On `login_finish`, server checks for `device_id` in the request. If present and valid (belongs to user), reuses it. Otherwise creates a new device. Device ID returned in response and stored on client disk. `list_devices` and `remove_device` handlers for device management.
|
|
|
|
### Simplified Pairing (Multi-Device)
|
|
`authorize_device()` only exports RSA + identity key (no sessions/sender keys). New device generates its own SPK + OPKs on first login, creates independent sessions via X3DH. Old messages readable via self-encryption (shared identity key). `reencrypt_history()` still runs to ensure all messages have self-encrypted copies.
|
|
|
|
### Real-time Conversation Notifications
|
|
`handle_create_conversation`, `handle_add_member`, `handle_remove_member`, `handle_leave_group`, `handle_delete_conversation`, `handle_accept_invitation` push notifications to affected members via `connected_clients`. Types: `conversation_created`, `member_added`, `member_removed`, `group_invitation`. GUI handles these via `conversation_updated` signal -> refreshes conversation list.
|
|
|
|
### Connection State + Auto-Reconnect
|
|
`ChatClient.connected` flag tracks TCP connection state. `_background_listener` sets `connected = False` when server closes connection and **fails all pending futures** with `ConnectionError` (prevents `send_and_recv` from hanging forever). `send_and_recv` has a 30s timeout via `asyncio.wait_for` and catches `ConnectionError`/`TimeoutError`. `reconnect()` re-establishes TCP + RSA challenge-response using in-memory keys (no password needed, includes `device_id`). GUI `_notification_loop` detects listener death -> triggers `_auto_reconnect` with exponential backoff (1s->2s->4s->...->30s). Connection indicator dot: green (connected), red (disconnected), orange (reconnecting).
|
|
|
|
### Server Per-Message Error Handling
|
|
Server dispatch loop wraps each handler call in individual try/except. Handler crashes return "Internal server error" response instead of killing the entire connection. Errors logged with `exc_info=True` for full tracebacks. GUI `_do_send_message`/`_do_find_or_create_and_send` catch exceptions and emit error signal (prevents silent hang when send fails).
|
|
|
|
### Online/Offline Status
|
|
- `db.get_user_contacts(user_id)` returns all user IDs sharing at least one conversation
|
|
- On login (`handle_login_finish`): server sends `online_users` list to new user + `user_online` to all contacts (only if user was fully offline before)
|
|
- On disconnect (`handle_client` finally block): if last connection drops, server sends `user_offline` to all contacts
|
|
- `_background_listener` routes `user_online`, `user_offline`, `online_users` to notification queue
|
|
- GUI: `_online_users: set[str]` tracks online users, green dot overlay on circular avatar in DM conversation list + green circle emoji in group info member list
|
|
|
|
### Leave Group
|
|
- `handle_leave_group` in server.py: validates membership, blocks DM leave (len<=2 and no name), transfers creator to first remaining member if creator leaves, removes member, notifies remaining via `member_removed`
|
|
- `ChatClient.leave_group()`: sends request, cleans up local sender key states on success
|
|
- GUI: red "Leave Group" button in group info dialog, confirmation dialog, resets view on success
|
|
|
|
### Delete Conversation
|
|
- **DMs:** Any member can delete. Only removes the deleting user from `conversation_members`. If both users delete, 0 members remain → conversation + files cleaned up.
|
|
- **Groups:** Only the creator (admin) can delete. Removes ALL members, cleans up `.enc` files from disk, deletes conversation via CASCADE.
|
|
- Server notifies remaining members via `member_removed` push.
|
|
- GUI: trash icon button in conversation header. Visible for DMs always, for groups only when user is creator.
|
|
- `chat_core.py`: cleans up local sender key states after successful delete.
|
|
|
|
### Group Invitations
|
|
- **Flow:** `create_conversation` (group) or `add_member` → creates invitation → pushes `group_invitation` notification → invitee sees in invitation list → Accept (adds to members, notifies) / Decline (deletes invitation)
|
|
- **DMs are unaffected:** `create_conversation` for DMs still auto-adds both members
|
|
- **DB:** `group_invitations` table with UNIQUE(conversation_id, user_id) to prevent duplicates
|
|
- **Server:** `handle_accept_invitation` verifies invitation exists, adds member, deletes invitation, notifies existing members via `member_added`. `handle_decline_invitation` just deletes.
|
|
- **GUI:** `inv_list` QListWidget (max 120px, amber border) above `conv_list`. Right-click → Accept/Decline. `invitation_received` signal triggers refresh + notification banner.
|
|
- **Routing fix (IMPORTANT):** `group_invitation` must be in the notification types list in `chat_core.py:_background_listener` (~line 304). Without it, invitations get routed to `_response_queue` and the GUI never sees them.
|
|
|
|
### Group Avatars
|
|
- Stored as files in `UPLOAD_DIR/avatars/group_{conv_id}.{ext}` (PNG or JPEG detected from magic bytes)
|
|
- `conversations.avatar_file` column stores the filename
|
|
- `list_conversations` response includes `avatar_file` so GUI knows which groups have avatars
|
|
- GUI: `_group_avatar_cache` dict, `_get_conv_avatar()` returns group avatar icon or default letter circle
|
|
- Group Info dialog shows 64px circular avatar + "Change Avatar" button (creator only)
|
|
- Periodic refresh every 2 minutes re-downloads all known group avatars
|
|
|
|
### File Sharing
|
|
- Reuses image upload/download infrastructure (`upload_image_start/chunk/end`, `download_image`)
|
|
- `upload_image_start` accepts optional `file_type` param: `"image"` (MAX_IMAGE_BYTES=5MB) or `"file"` (MAX_FILE_BYTES=50MB)
|
|
- `ChatClient.send_file()`: reads raw file, AES-256-GCM encrypts, chunked upload, sends message with `file` field in payload (`{file_id, aes_key, iv, filename, size, mime_type}`)
|
|
- `ChatClient.download_file()`: identical to `download_image()` — chunked download + AES-GCM decrypt
|
|
- GUI: attach button is dropdown menu (Image / File), file messages render as styled cards with paperclip icon (transparent background, border) and clickable download link (`file://{file_id}`), context menu "Download file" option
|
|
- Files stored as `.enc` in UPLOAD_DIR, same as images
|
|
|
|
### Unread Count Badges
|
|
- `_unread_counts: dict[str, int]` replaces old `_unread_convs: set`
|
|
- `_on_notification()` increments count per conversation
|
|
- `_on_conv_selected()` clears count for selected conversation
|
|
- Display: `(3) Username` with bold font, instead of old `● Username`
|
|
|
|
### User Profiles
|
|
`user_profiles` table separated from `users` (clean separation, users = auth only). Default profile created on registration (`db.create_default_profile`). Visibility rules applied server-side in `db.get_user_profile(viewer_id)`. Avatars stored as files in `UPLOAD_DIR/avatars/{user_id}.{ext}` (not in DB). Format detection from magic bytes (PNG header vs default JPEG). UserProfileDialog shows circular cropped avatar (QPainter).
|
|
|
|
### Prekey Replenishment + SPK Rotation
|
|
After login, `_ensure_prekeys()` is called as a background task. Checks two things:
|
|
1. **OPK count** — if < 20, generates and uploads a new batch of 50
|
|
2. **SPK age** — server returns `spk_created_at` in `get_prekey_count` response. If SPK is >= 7 days old (`SPK_ROTATION_DAYS`), triggers rotation: saves current SPK as `prev_spk_private.bin`/`prev_spk_id.txt` (grace period), generates new SPK, uploads to server.
|
|
|
|
### Password-Based Key Encryption (ECP1 Format) — M3
|
|
Private keys (RSA, Ed25519) are encrypted with a custom envelope instead of `BestAvailableEncryption`:
|
|
- **Key derivation:** PBKDF2-HMAC-SHA256 with 600,000 iterations (OWASP 2023 compliant)
|
|
- **Encryption:** AES-256-GCM with the derived key, magic bytes as AAD
|
|
- **Format:** `_ECP1_MAGIC("ECP1", 4B) + salt(16B) + nonce(12B) + ciphertext_with_tag(N+16B)`
|
|
- **Backward compatibility:** `load_private_key()` and `load_ed25519_private()` detect ECP1 magic prefix. If absent, fall back to legacy PEM parsing (old `BestAvailableEncryption` format). On next save, files are re-encrypted in ECP1 format.
|
|
- **Functions:** `_encrypt_private_key()`, `_decrypt_private_key()` in `crypto_utils.py`
|
|
- **Applied to:** `serialize_private_key()` (RSA), `serialize_ed25519_private()` (Ed25519)
|
|
|
|
### SPK Rotation (7-Day Cycle) — M4
|
|
Signed Pre-Keys rotate periodically to limit exposure from a compromised SPK:
|
|
- **Rotation interval:** `SPK_ROTATION_DAYS = 7` (constant in `chat_core.py`)
|
|
- **Trigger:** `_ensure_prekeys()` checks `spk_created_at` from `get_prekey_count` response. If age >= 7 days, calls `_generate_and_upload_prekeys()`.
|
|
- **Grace period:** Before generating a new SPK, the current one is saved as `prev_spk_private.bin` / `prev_spk_id.txt`. Loaded on login into `_prev_spk_private` / `_prev_spk_id`.
|
|
- **Fallback on decrypt:** When `_decrypt_dm()` processes an X3DH header and decryption fails with the current SPK, it retries with the previous SPK via `_process_x3dh_header(..., spk_override=self._prev_spk_private)`. This handles in-flight X3DH initiated before rotation.
|
|
- **Server side:** `get_signed_prekey()` in `db.py` returns `created_at` column. `handle_get_prekey_count` includes `spk_created_at` (ISO format) in response.
|
|
- **Server SPK replacement:** `store_signed_prekey()` deletes old SPK and inserts new one — only one active SPK per device on server.
|
|
|
|
### Ratchet State Rollback on Decrypt Failure — M9
|
|
Both `DoubleRatchet.decrypt()` and `SenderKeyState.decrypt()` modify internal state (chain keys, counters, DH keys) before attempting AES-GCM decryption. If decryption fails (corrupted data, wrong key, AAD mismatch), the state would be permanently corrupted.
|
|
|
|
**DoubleRatchet fix:**
|
|
- `_snapshot()` captures all mutable fields: `dh_pair`, `dh_remote`, `root_key`, `send_chain_key`, `recv_chain_key`, `send_n`, `recv_n`, `prev_send_n`, `skipped` dict (shallow copy)
|
|
- `decrypt()` takes snapshot before any state modification, wraps the entire DH ratchet + chain advance + AES-GCM decrypt in try/except, calls `_restore()` on failure
|
|
- Special case: skipped message decryption (no state modification needed) — if AES-GCM fails, the popped key is restored to `skipped` dict
|
|
|
|
**SenderKeyState fix:**
|
|
- Before fast-forwarding the chain, snapshots `chain_key`, `n`, `_known_keys` (shallow copy)
|
|
- On any exception during fast-forward or AES-GCM decrypt, all three are restored
|
|
|
|
### Rate Limits
|
|
- Per-IP+email window (60s): register 3/min, login 10/min, send_message 20/min
|
|
- Per-connection: 20 req/s
|
|
- Per-IP: max 10 connections, global max 200
|
|
- Pairing: TTL 120s, max 90 poll attempts, pairing_start 10/min, pairing_poll 120/min, client polls every 2s
|
|
|
|
### GUI Font Handling (IMPORTANT)
|
|
All widget stylesheet `font-size` declarations use `pt` (not `px`). Using `px` in Qt stylesheets sets `pixelSize` and leaves `pointSize=-1`, which causes `QFont::setPointSize: Point size <= 0` warnings on Windows. Conversion: `pt ~= px * 0.75` at 96 DPI. HTML styles inside QTextBrowser (`_render_single_message_html`) still use `px` — that's fine, QTextBrowser uses its own HTML renderer. Bold fonts for list items use `_bold_font()` helper + `item.setData(FontRole)` to avoid the same issue.
|
|
|
|
### Phantom Users (Anti User-Enumeration)
|
|
- When a user creates a conversation with an unregistered email, the server creates a "phantom" user with `rsa_public_key = 'PHANTOM'` marker
|
|
- Phantom users have real crypto keys (Ed25519 IK, X25519 SPK + 5 OPKs) so X3DH works on the client side
|
|
- `handle_find_conversation` and `handle_create_conversation` create phantoms instead of returning "User not found"
|
|
- `handle_send_message` skips phantom recipients when storing `message_recipients` — only sender's self-encrypted copy is saved
|
|
- `phantom_user_ids: set[str]` in-memory cache loaded at startup from DB, updated on create/delete
|
|
- On registration (`handle_register_confirm`): if email belongs to a phantom, the phantom is **upgraded in-place** via `db.upgrade_phantom_user()` — preserves user_id and all FK references (invitations, conversation_members). Phantom's server-generated prekeys are deleted (real user uploads own).
|
|
- `handle_create_conversation` (groups) and `handle_add_member` create invitations for phantom users too. Push notifications only sent to non-phantom users. When phantom registers and logs in, they see pending invitations.
|
|
- Messages sent to phantom users are NOT stored and NOT recoverable after registration — this is by design (prevents user enumeration, sender sees own messages via self-encryption)
|
|
- DB functions: `db.create_phantom_user(email)`, `db.is_phantom_user(user_id)`, `db.delete_phantom_user(user_id)`, `db.upgrade_phantom_user(phantom_id, username, rsa_public_key_pem, identity_key)`, `db.get_all_phantom_user_ids()`
|
|
|
|
### Logout/Login Fix
|
|
- `_is_logout` flag in MainWindow prevents `closeEvent()` from calling `bridge.stop()` which killed the asyncio loop
|
|
- On logout: set `_is_logout = True`, call `bridge.logout()`, then `close()`
|
|
- `closeEvent()` only calls `bridge.stop()` if `not self._is_logout`
|
|
- This allows `main()` to re-create the login/main windows after logout
|
|
|
|
### Server Graceful Shutdown
|
|
- SIGINT handler force-closes all writers in `connected_clients` before the asyncio server context manager exits
|
|
- Without this, `async with server:` waited forever for `handle_client` loops to finish
|
|
|
|
### Version Negotiation
|
|
- `VERSION = "0.8"` constant in `protocol.py` (shared between client and server)
|
|
- Client sends `client_version` in `login_finish` request (both `login()` and `reconnect()`)
|
|
- Server logs `client_version`, returns `server_version` in `login_finish` response
|
|
- Server startup log includes version: `"Encrypted chat server v0.8 listening on ..."`
|
|
- Future: server can reject incompatible client versions, client can warn about outdated server
|
|
|
|
## Conventions
|
|
|
|
- Server handlers: `handle_<type>(msg, session, writer)` — registered in dispatch table in `handle_client()`
|
|
- DB functions: one `get_connection()` per call, `cursor(dictionary=True)`, returns dicts
|
|
- Binary data: always base64 in protocol (`encode_binary`/`decode_binary`)
|
|
- GUI signals: bridge emits `pyqtSignal`, MainWindow connects in `_connect_signals()`
|
|
- Error responses: `{"status": "error", "data": {"message": "..."}}`
|
|
- Notification decrypt returning `None` = control message, skip silently
|
|
- GUI stylesheet font sizes: always `pt`, never `px` (see Font Handling section above)
|
|
- File sharing reuses image upload infrastructure with `file_type` parameter
|
|
- Avatar files stored in `UPLOAD_DIR/avatars/` — user: `{user_id}.{ext}`, group: `group_{conv_id}.{ext}`
|
|
|
|
## Aktuální stav práce
|
|
|
|
### ✅ Dokončeno (tato session)
|
|
- Logout/login bug fix — `_is_logout` flag prevents bridge.stop() on logout
|
|
- Hover text readability — `color: #cdd6f4;` added to `QListWidget::item:hover`
|
|
- File card background — `background:transparent; border:1px solid #45475a`
|
|
- Delete conversations — full stack (db, server, chat_core, gui), DMs + groups (creator-only), file cleanup from disk
|
|
- Group invitation system — full stack (schema, db, server, chat_core, gui), create/accept/decline, real-time notification, invitation list UI
|
|
- Circular avatars in conversation list — QPainter circular crop, default letter avatars, online green dot overlay
|
|
- Group avatar support — upload/download, display in group info dialog, "Change Avatar" button (creator only)
|
|
- Server graceful shutdown — force-close connected clients on SIGINT
|
|
- Profile dialog avatar circular crop — QPainter in UserProfileDialog._on_avatar_loaded
|
|
- Periodic refresh timer — 2-minute QTimer re-downloads avatars + invitations
|
|
- Group invitation notification fix — `group_invitation` added to `_background_listener` notification types
|
|
- Delete button in conversation header — trash icon for DMs always, groups creator-only
|
|
- File cleanup on conversation delete — `db.get_conversation_file_ids()` + unlink `.enc` files
|
|
- Removed right-click "Delete conversation" from conversation list context menu
|
|
- README.md updated
|
|
- **H4 Race conditions fix** — 4 asyncio.Lock guards (`_clients_lock`, `_conn_lock`, `_pairing_lock`, `_uploads_lock`) pro všechny sdílené mutable struktury v server.py. `_notify_users()` + `_notify_users_individual()` helpery. Rate limit memory cleanup v periodic task. Všechny I/O operace mimo kritické sekce.
|
|
- **Unread counts pro offline uživatele** — `db.get_unread_counts()` dotaz přes `message_reads` + `message_recipients`, server vrací `unread_count` v `list_conversations`, GUI populuje `_unread_counts` ze serverových dat (max z server vs local). Opravuje bug kdy offline uživatel po přihlášení neviděl nepřečtené zprávy.
|
|
- **C6 Path traversal fix** — `_UUID_RE` regex + `_valid_file_id()`, `_safe_upload_path()`, `_safe_avatar_path()` helpery v server.py. UUID validace v `handle_upload_image_start`, `handle_download_image`. `is_relative_to()` guard ve všech path konstrukcích: upload start/end, download, delete_message file cleanup, delete_conversation file cleanup, _cleanup_uploads, get/update avatar, get/update group avatar. Celkem 10 guardovaných míst.
|
|
- **C3+H1+M13 Lokální šifrování + permissions** — `derive_local_storage_key()` v crypto_utils.py (HKDF z identity key, odlišný salt/info od self-encryption key). `_encrypt_local()`/`_decrypt_local()` helpery v chat_core.py (AES-256-GCM, formát: nonce(12)+tag(16)+ct). `_save_session`/`_load_session`, `_save_sender_key_state`/`_load_sender_key_state`, `_save_recv_sender_key`/`_load_recv_sender_key` — volitelný `local_key` parametr, při nastavení šifruje/dešifruje, `chmod 0o600` na soubory. `ChatClient._local_key` derivováno při login/registraci/pairingu. Transparentní migrace: pokud dešifrování selže, zkusí plaintext a re-uloží šifrovaně. `os.chmod(d, 0o700)` na všechny `mkdir()` v get_key_dir, opk_private, sessions, sender_keys, sender_keys_recv, message_cache. `os.chmod(p, 0o600)` na plaintext fallback message cache.
|
|
- **H7 Avatar path traversal** — `_safe_avatar_path()` guard na handle_get_avatar, handle_get_group_avatar + defense-in-depth na handle_update_avatar, handle_update_group_avatar.
|
|
- **Multi-device support (per-device sessions)** — `devices` table, `device_id` columns on prekeys/messages/recipients/sender_keys. Server: device registration at login, `writer_device_map`, per-device key bundles (`device_bundles` array), per-device notification routing (`device_entries`), `list_devices`/`remove_device` handlers. Client: `device_id` persistence, sessions keyed by `"user_id:device_id"`, `_get_device_bundles()` with 5-min TTL cache, per-device encryption in `_send_dm`/`send_image`/`send_file`/`_distribute_sender_key`, `sender_device_id` in decrypt routing, `decrypt_notification()` handles `device_entries` format. Pairing simplified: only RSA + identity key transfer, new device generates own SPK + OPKs. Migration: old session files auto-migrated, backward compat with old clients/servers. H12 OPK race condition fixed (SELECT FOR UPDATE).
|
|
- **Connection resilience fixes** — `_background_listener` fails all pending futures with `ConnectionError` on disconnect (prevents hang). `send_and_recv` has 30s timeout + catches `ConnectionError`. Server dispatch has per-message try/except (handler crash no longer kills connection). GUI `_do_send_message`/`_do_find_or_create_and_send` catch exceptions and emit error signal.
|
|
- **DB transaction fix** — `db.get_key_bundles_for_user()` had "Transaction already in progress" error because mysql-connector starts implicit transactions. Fixed with `conn.commit()` before `conn.start_transaction()`.
|
|
- **H5+H6 Protocol error handling** — `decode_binary()` catches `binascii.Error` → `ValueError`. `parse_message()` catches `JSONDecodeError`/`UnicodeDecodeError` → `ValueError`. Server dispatch already handles `ValueError` from `read_message()` gracefully.
|
|
- **H3+H13 Anti-enumeration** — `handle_register_start` returns same "ok" response for existing email (no "Email already in use" leak). `handle_login_start` returns fake challenge for non-existent email. `handle_login_finish` returns generic "Invalid credentials" for all failure cases. `get_user_info` moved behind auth barrier (requires login).
|
|
- **H8 Password memory cleanup** — `register()`, `login()`, `pairing_wait()` convert password to `bytearray`, zero out in `finally` block after key derivation.
|
|
- **H10 Image validation** — `_safe_load_image()` helper validates size (<10MB) and dimensions (<8192px) before `QImage.fromData()`. Applied to all 6 image loading locations in gui_client.py.
|
|
- **H11 Filename sanitization** — `_safe_filename()` helper strips path components via `os.path.basename()`. Applied to save dialogs and image dialog title.
|
|
- **C1+C2+C5 DoS hardening** — C1: `LimitOverrunError` now drains buffer and raises `ValueError` (server sends error response instead of silent disconnect; memory already protected by `limit=` on StreamReader). C2: `MAX_SENDER_KEY_SKIP` reduced from 1000 to 256 (matches DoubleRatchet `MAX_SKIP`). C5: `handle_upload_image_end` validates `received_bytes == file_size` before completing upload. M12 (upload end size validation) also fixed by C5.
|
|
- **M2+M8+M10+M11 Security hardening batch** — M2: SenderKeyState HKDF salt changed from `b""` to `b"\x00" * 32` (matches X3DH convention). M8: `_valid_file_id()` renamed to `_valid_uuid()`, UUID validation added to all handlers accepting client-provided `conv_id`, `user_id`, `message_id`, `device_id`. M10: `handle_mark_read` caps `message_ids` to 500 (prevents slow SQL DoS). M11: `handle_pairing_start` generates `poll_token` (secrets.token_hex(16)), `handle_pairing_poll` requires and validates it via `secrets.compare_digest()` — prevents unauthorized poll/payload extraction.
|
|
- **H2+H14 TLS hardening** — `TLS_INSECURE` a `TLS_AUTOGEN` nyní vyžadují `ENVIRONMENT=dev` (RuntimeError bez toho). Warning log na serveru i klientovi když TLS vypnuté. C4 (OPK file permissions) bylo již opraveno v C3+H1+M13 batchi.
|
|
- **Online dot fix + sorting** — Fixed timing issue where `online_users` signal processed before conv list populated. `_rebuild_conv_list()` sorts: favorites first → online DMs → rest alphabetically.
|
|
- **Favorites system** — Right-click context menu on conversation list → Add/Remove from favorites. Star indicator (★). Persisted to `favorites.json` in user key directory.
|
|
- **Group renaming** — Full stack: `db.update_conversation_name()`, `handle_rename_conversation` (server, creator-only, max 100 chars), `rename_conversation()` (chat_core), "Rename" button in group info dialog (GUI), `conversation_renamed` push notification to all members.
|
|
- **M3+M4+M9 Security hardening** — M3: PBKDF2 600k iterations (`_encrypt_private_key`/`_decrypt_private_key` s ECP1 formátem, backward compat pro PEM). M4: SPK rotace každých 7 dní, `spk_created_at` v `get_prekey_count`, grace period s `prev_spk_private.bin`, fallback v `_process_x3dh_header`/`_decrypt_dm`. M9: `_snapshot()`/`_restore()` v `DoubleRatchet.decrypt()`, snapshot/restore v `SenderKeyState.decrypt()`.
|
|
- **Phantom invitation fix** — Phantom users now receive group invitations. `handle_create_conversation` and `handle_add_member` create invitations for phantoms (no push notification). `handle_register_confirm` upgrades phantom in-place via `db.upgrade_phantom_user()` (preserves user_id + FK references). `handle_add_member` creates phantom for unregistered emails (same as `create_conversation`). After registration, user sees pending invitations on login.
|
|
- **Message Search** — Client-side search through decrypted message cache. `ChatClient.search_messages()` searches local cache. GUI: collapsible search bar (Ctrl+F) with prev/next navigation, match count, yellow/orange highlighting in message HTML. Escape closes search. Search button in chat header.
|
|
- **Session Recovery** — `session_reset` protocol message + `handle_session_reset` server handler. `ChatClient.reset_session()` deletes local session + notifies peer. Peer handles `session_reset` notification by deleting their session. Next message auto-creates new session via X3DH. GUI: "Reset session with sender" context menu on undecryptable messages, status bar notification on incoming reset.
|
|
- **L8 Phantom user DB inflation fix** — `_valid_email()` helper validates email format before phantom creation in `handle_find_conversation`, `handle_create_conversation`, `handle_add_member`. `db.cleanup_stale_phantoms(30)` deletes phantom users older than 30 days with no active conversations with real users. Runs in `_periodic_cleanup` every 10 minutes, refreshes in-memory `phantom_user_ids` cache.
|
|
- **M6 TOCTOU race fix** — `db.remove_conversation_member_atomic()` returns bool (True if row existed). Used in `handle_remove_member` (checks return value, returns error if already removed) and `handle_leave_group`. Defense-in-depth: pre-checks remain for user-friendly errors, atomic operation prevents double-removal.
|
|
|
|
### 🐛 Známé bugy a problémy
|
|
- **Sender Key Redistribution (High Priority):** New group member can't decrypt old messages. On `add_member`, existing members should re-create and redistribute sender keys.
|
|
- **Database Connection Pooling:** Every `db.*` call creates new MySQL connection. Should use pooling for production.
|
|
- **Group delete confirmation message is generic** — could say "Delete group and remove all members?" for groups vs "Delete conversation?" for DMs.
|
|
|
|
### ⏭️ Další kroky (TODO)
|
|
|
|
#### Bezpečnostní opravy (priorita dle auditu)
|
|
1. **C6 (CRITICAL): Path traversal přes file_id** — `handle_upload_image_start` vytváří soubor `UPLOAD_DIR / f"{file_id}.tmp"` bez validace. Útočník může `../../...` a zapisovat/mazat mimo UPLOAD_DIR. Řešení: validovat UUID formát, ověřit `path.resolve().is_relative_to(UPLOAD_DIR.resolve())`.
|
|
2. ~~**H12 (HIGH): OPK race condition v db.get_key_bundle()**~~ ✅ OPRAVENO (součást multi-device — SELECT FOR UPDATE v consume_one_time_prekey + get_key_bundle)
|
|
3. **H3+H13: User enumeration** — `get_user_info` dostupné bez auth, vrací identity_key pro libovolný email. `register_start`/`login_start` vrací jednoznačné chyby. Řešení: auth pro `get_user_info`, generické odpovědi pro register/login.
|
|
4. ~~**H2+H14: TLS hardening**~~ ✅ OPRAVENO — `TLS_INSECURE` a `TLS_AUTOGEN` vyžadují `ENVIRONMENT=dev`. Warning log při vypnutém TLS.
|
|
5. ~~**C1+C2+C5**~~ ✅ OPRAVENO — DoS vektory (LimitOverrunError → ValueError, MAX_SENDER_KEY_SKIP 256, upload completeness check)
|
|
6. **C3+C4+H1** — Šifrování dat na disku (message cache, sessions, OPK permissions, `chmod 0o700` pro adresáře)
|
|
7. **H5+H6** — Error handling v protokolu (base64, JSON)
|
|
8. **H7** — Path traversal v avatar souborech (`resolved_path.is_relative_to()`)
|
|
9. ~~**M11 (MEDIUM): Pairing poll DoS**~~ ✅ OPRAVENO — poll_token binding (secrets.token_hex(16) + secrets.compare_digest)
|
|
10. ~~**M12: Upload end bez validace velikosti**~~ ✅ OPRAVENO (součást C5 fixu — `handle_upload_image_end` validuje `received_bytes == file_size`)
|
|
11. ~~**L8: Phantom user DB inflation**~~ ✅ OPRAVENO — email validace + periodic cleanup stale phantoms (30 dní)
|
|
12. **Version negotiation** — `VERSION = "0.8"` v protocol.py, klient posílá `client_version` při loginu, server loguje a vrací `server_version`
|
|
|
|
#### Před nasazením do produkce (checklist)
|
|
1. **TLS certifikáty** — Získat certifikát (Let's Encrypt / vlastní CA). Nastavit `TLS_ENABLED=true`, `TLS_CERT_FILE`, `TLS_KEY_FILE` v `.env`. Ověřit že `TLS_INSECURE` a `TLS_AUTOGEN` NEJSOU nastavené (vyžadují `ENVIRONMENT=dev`). Na klientovi nastavit `TLS_ENABLED=true` a případně `TLS_CA_FILE` pokud vlastní CA.
|
|
2. **Email validace** — Zapnout `_valid_email()` kontrolu v `handle_find_conversation`, `handle_create_conversation`, `handle_add_member` (kód existuje v server.py, volání zakomentována). Teď vypnuto protože dev prostředí používá emaily bez @.
|
|
3. **MySQL TLS** — Přidat SSL parametry do `db.get_connection()` (`ssl_ca`, `ssl_cert`, `ssl_key`) pokud DB běží na jiném stroji.
|
|
4. **Connection pooling** — Nahradit `get_connection()` za `mysql.connector.pooling.MySQLConnectionPool(pool_size=10)`.
|
|
5. **SMTP** — Nastavit reálný SMTP server pro registrační kódy (`SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`, `SMTP_FROM`).
|
|
6. **UPLOAD_DIR** — Ověřit že `UPLOAD_DIR` je na persistentním disku s dostatkem místa, správnými právy (0o700).
|
|
7. **Rate limity** — Přezkoumat limity pro produkční zátěž (registrace 3/min, login 10/min, send_message 20/min, max 10 spojení/IP).
|
|
8. **Packaging** — Zabalit klienta (pyinstaller / cx_Freeze) pro distribuci. Po zabalení zvážit auto-update mechanismus a `get_version` endpoint.
|
|
9. **Penetrační testy** — Provést před ostrým nasazením (viz sekce níže).
|
|
10. **Backup** — Nastavit pravidelný backup MySQL databáze + `UPLOAD_DIR`.
|
|
|
|
#### Penetrační testy
|
|
- Naplánovat a provést manuální penetrační testy zaměřené na:
|
|
- Path traversal (file_id, avatar_file)
|
|
- DoS vektory (readuntil, sender key fast-forward, upload flooding)
|
|
- Race conditions (OPK reuse, membership TOCTOU)
|
|
- User enumeration (register, login, get_user_info)
|
|
- TLS downgrade / MITM bez TLS
|
|
- Pairing session hijacking
|
|
- Memory exhaustion (rate_limits, phantom users, message_ids)
|
|
- Vytvořit testovací skripty pro automatizované security testy
|
|
- Zdokumentovat výsledky a opravit nalezené problémy
|
|
|
|
#### Ke zvážení
|
|
- **Auto-update klientů** — distribuce aktualizovaných souborů klientům před login/registrací. Řešit až po kompilaci/packagingu (pyinstaller apod.). Mechanismus: server verze check → klient stáhne nové soubory → restart.
|
|
- **Server version check endpoint** — po packagingu mít jednoduchý endpoint (např. `get_version`), který vrací min/aktuální podporovanou verzi klienta + URL/metadata pro update; klient může před loginem ověřit kompatibilitu a nabídnout update. Vhodné i pro postupné vypínání starých klientů.
|
|
|
|
#### Funkční vylepšení
|
|
1. **Sender Key Redistribution** — on add_member, redistribute sender keys to all members including new one
|
|
2. ~~**Device Linking fix**~~ ✅ — replaced with true multi-device (per-device sessions, simplified pairing)
|
|
3. ~~**SPK Rotation**~~ ✅ — periodic rotation with grace period (implemented in M4 fix)
|
|
4. **Typing Indicators** — `typing_start`/`typing_stop` protocol + GUI indicator
|
|
5. ~~**CLI support**~~ ✅ — profiles, file sharing, invitations, leave/rename/delete, search, devices in `client.py`
|
|
6. ~~**Message search**~~ ✅ — client-side search through decrypted cache, Ctrl+F toggle, highlight + navigation
|
|
7. ~~**Session Recovery**~~ ✅ — `session_reset` protocol, auto-recreate via X3DH on next message
|
|
8. **Connection Pooling** — `mysql.connector.pooling` for production
|
|
9. ~~**Version negotiation**~~ ✅ — `VERSION = "0.8"` in protocol.py, client sends `client_version` at login, server logs it and returns `server_version`
|
|
|
|
## Bezpečnostní audit (Security Audit)
|
|
|
|
Kompletní audit provedený přes všechny soubory projektu. Nálezy seřazené podle závažnosti.
|
|
|
|
### 🔴 CRITICAL — Okamžitě řešit před nasazením
|
|
|
|
#### ~~C1. readuntil() bez limitu → memory exhaustion (protocol.py:62)~~ ✅ OPRAVENO
|
|
`ProtocolReader.read_message()` volá `readuntil(b"\n")`, které načte CELOU zprávu do paměti PŘED kontrolou velikosti. Útočník pošle gigabyty dat bez newline → server spadne na out-of-memory.
|
|
```python
|
|
line = await self._reader.readuntil(b"\n") # buffers everything first!
|
|
if len(line) > MAX_MESSAGE_BYTES: # too late
|
|
```
|
|
**Řešení:** Implementovat framing s hlavičkou obsahující velikost zprávy, nebo použít `readuntil()` s `limit` parametrem (asyncio StreamReader nemá nativně — nutno obalit vlastním čtením po částech).
|
|
|
|
#### ~~C2. SenderKeyState — neomezený fast-forward DoS (crypto_utils.py:642-645)~~ ✅ OPRAVENO
|
|
Při dešifrování skupinové zprávy s libovolně vysokým `n` se smyčka `while self.n <= n` provede milionkrát — derivuje milion klíčů, spotřebuje stovky MB RAM.
|
|
```python
|
|
while self.n <= n:
|
|
self.chain_key, mk = kdf_ck(self.chain_key)
|
|
self._known_keys[self.n] = mk # unbounded dict growth
|
|
self.n += 1
|
|
```
|
|
**Řešení:** Přidat `MAX_FORWARD_SKIP` limit (např. 1000) — stejně jako Double Ratchet má `MAX_SKIP=256`.
|
|
|
|
#### C3. Dešifrované zprávy uložené jako plaintext na disku (chat_core.py:222-239)
|
|
Message cache v `~/.encrypted_chat/{email}/message_cache/{conv_id}.json` obsahuje plný obsah dešifrovaných zpráv v nešifrovaném JSONu. Bez nastavení `chmod 0o600`. Kdokoliv s přístupem k disku přečte kompletní historii.
|
|
**Řešení:** Šifrovat cache klíčem odvozeným z identity key + nastavit `chmod 0o600` na soubory.
|
|
|
|
#### ~~C4. OPK private keys bez file permissions (chat_core.py:153-156)~~ ✅ OPRAVENO
|
|
OPK privátní klíče se ukládají bez `os.chmod(0o600)`. RSA klíče (řádek 87) a identity key (řádek 121) mají `chmod` — OPK ne. Na sdílených systémech může jiný uživatel přečíst ephemeral klíče.
|
|
**Opraveno:** Součást C3+H1+M13 fixu — `_save_opk_private()` nyní volá `os.chmod(path, 0o600)` + `os.chmod(dir, 0o700)`.
|
|
|
|
#### ~~C5. Chunked upload nevaliduje celkovou velikost (server.py:1138-1142)~~ ✅ OPRAVENO
|
|
`handle_upload_image_chunk` akumuluje `received_bytes` ale nekontroluje limit. Útočník deklaruje `file_size=5MB`, pak posílá chunky donekonečna → disk exhaustion.
|
|
```python
|
|
upload["received_bytes"] += len(raw) # no check against file_size!
|
|
```
|
|
**Řešení:** Přidat `if upload["received_bytes"] > upload["file_size"]: reject`.
|
|
|
|
### 🟠 HIGH — Řešit před production nasazením
|
|
|
|
#### H1. Session + sender key soubory nešifrované na disku (chat_core.py:176-215)
|
|
Double Ratchet session state (DH privátní klíče, root key, chain keys) a sender key state se ukládají jako plaintext hex JSON v `sessions/` a `sender_keys/`. Bez šifrování, bez `chmod 0o600`. Kompromitace disku = dešifrování celé historie.
|
|
**Řešení:** Šifrovat state klíčem z identity key, nastavit `chmod 0o600`.
|
|
|
|
#### ~~H2. TLS vypnuté ve výchozím stavu (chat_core.py:274-291, server.py)~~ ✅ OPRAVENO (hardening)
|
|
`TLS_ENABLED` je defaultně `false`. Bez TLS jdou po síti RSA challenge-response, session tokeny a metadata v plaintextu. `TLS_INSECURE=true` vypíná certificate verification → MITM.
|
|
**Opraveno:** `TLS_INSECURE` a `TLS_AUTOGEN` nyní vyžadují `ENVIRONMENT=dev` — v produkci RuntimeError. Warning log při vypnutém TLS na serveru i klientovi. TLS_ENABLED zůstává default false (uživatel nemá certifikát), ale po nasazení Let's Encrypt stačí flip na true.
|
|
|
|
#### H3. User enumeration přes registraci (server.py:182-189)
|
|
Registrace vrací "Email already in use" pro existující uživatele vs. tiché vytvoření phantoma pro neexistující. Útočník může enumerovat platné emaily.
|
|
**Řešení:** Vrátit generickou odpověď "Check your email for verification code" i když email existuje.
|
|
|
|
#### ~~H4. Race conditions v in-memory strukturách (server.py: multiple)~~ ✅ OPRAVENO
|
|
`connected_clients` dict, `phantom_user_ids` set, `pairing_sessions` dict — čteny a zapisovány z více concurrent koroutin bez synchronizace. Asyncio je single-threaded, ale yieldy uvnitř handlerů (await) mohou způsobit nekonzistentní stav.
|
|
**Opraveno:** 4 asyncio.Lock guards: `_clients_lock` (connected_clients, phantom_user_ids), `_conn_lock` (connection_counts, current_connections, rate_limits), `_pairing_lock` (pairing_sessions, pending_registrations), `_uploads_lock` (pending_uploads). Helper funkce `_notify_users()` / `_notify_users_individual()` — snapshot under lock, send outside. Rate limit memory cleanup v periodic task. Žádný handler nedrží dva locky současně → deadlock impossible.
|
|
|
|
#### H5. base64 decode bez error handling (protocol.py:14-16, server.py + chat_core.py)
|
|
`decode_binary()` volá `base64.b64decode()` bez try-except. Nevalidní base64 od klienta → unhandled `binascii.Error` → handler crash. Mnoho callsites v server.py (řádky 357, 378, 783) nemá catch.
|
|
**Řešení:** Obalit `decode_binary()` try-except, nebo validovat base64 vstup před dekódováním.
|
|
|
|
#### H6. JSON parsing bez exception handling (protocol.py:48-50)
|
|
`parse_message()` volá `json.loads()` bez try-catch. Malformovaný JSON = neošetřený `JSONDecodeError`. Server handler catch (řádek 1399) to odchytí, ale není to explicitní.
|
|
**Řešení:** Obalit `json.loads()` v `parse_message()` try-except s explicitní chybovou zprávou.
|
|
|
|
#### H7. Path traversal v avatar souborech (server.py:1265, 1318)
|
|
`avatar_file` ze serveru (z DB) se přímo joinuje s `UPLOAD_DIR / "avatars"` bez validace. Pokud DB obsahuje `../../etc/passwd`, server přečte libovolný soubor.
|
|
**Řešení:** Přidat `resolved_path.resolve().is_relative_to(UPLOAD_DIR)` check.
|
|
|
|
#### H8. Heslo v paměti jako Python string (chat_core.py, gui_client.py)
|
|
Python stringy jsou immutable — nelze je bezpečně vymazat z paměti. Heslo zůstává v paměti dokud garbage collector neuklidí. Memory dump = plaintext heslo.
|
|
**Řešení:** Použít `bytearray` (mutable), po použití přepsat nulami: `pwd[:] = b'\x00' * len(pwd)`.
|
|
|
|
#### H9. Self-encryption key je statický a deterministický (chat_core.py:904+, crypto_utils.py:224-233)
|
|
`derive_self_encryption_key(identity_private)` produkuje vždy stejný klíč. Kompromitace identity klíče = dešifrování všech vlastních kopií zpráv navždy. Žádná forward secrecy pro self-copies.
|
|
**Poznámka:** Toto je by-design (nutné pro cross-device čtení), ale je to architektonické omezení.
|
|
|
|
#### H10. Malicious image data → QImage crash (gui_client.py)
|
|
`QImage.fromData(data)` zpracovává nevalidované binární data. Speciálně vytvořený obrázek může způsobit crash, memory exhaustion, nebo v extrémním případě RCE přes Qt image codec.
|
|
**Řešení:** Validovat velikost dat před parsováním, limit na max rozlišení.
|
|
|
|
#### H11. Filename z serveru v save dialogu (gui_client.py:2389, 2460)
|
|
Server-controlled `filename` se předává jako default do `QFileDialog.getSaveFileName()`. Pokud server pošle `"../../../.bashrc"`, dialog to navrhne.
|
|
**Řešení:** Sanitizovat filename — odstranit `../`, `\`, absolutní cesty. Použít jen `os.path.basename()`.
|
|
|
|
### 🟡 MEDIUM — Zvážit pro hardening
|
|
|
|
#### M1. Inconsistentní Ed25519 serializace (crypto_utils.py:99-102)
|
|
Bez hesla: 32 raw bytes. S heslem: PEM PKCS8 (~302 bytes). Dva různé formáty mohou způsobit problém při migraci nebo obnově klíčů.
|
|
**Poznámka:** M3 fix částečně řeší — s heslem je nyní ECP1 formát (ne PEM), ale `load_ed25519_private()` stále detekuje 3 formáty (ECP1, PEM, raw). Legacy PEM soubory se automaticky migrují při dalším uložení.
|
|
|
|
#### ~~M2. Prázdný salt v SenderKeyState HKDF (crypto_utils.py:610)~~ ✅ OPRAVENO
|
|
`hkdf_derive(sender_key, salt=b"", ...)` — RFC 5869 doporučuje nenulový salt. X3DH správně používá `b"\x00" * 32`.
|
|
**Opraveno:** Změněno `salt=b""` → `salt=b"\x00" * 32` aby odpovídalo X3DH konvenci.
|
|
|
|
#### ~~M3. PBKDF2 iterace pod doporučeným minimem (crypto_utils.py)~~ ✅ OPRAVENO
|
|
`BestAvailableEncryption` používá ~100k iterací PBKDF2. OWASP 2023 doporučuje 480k+.
|
|
**Opraveno:** Nahrazeno vlastním `_encrypt_private_key`/`_decrypt_private_key` s PBKDF2-HMAC-SHA256 (600k iterací) + AES-256-GCM. ECP1 formát (magic prefix) s backward compat pro staré PEM soubory. Aplikováno na RSA (`serialize_private_key`/`load_private_key`) i Ed25519 (`serialize_ed25519_private`/`load_ed25519_private`).
|
|
|
|
#### ~~M4. SPK bez replay protection a bez rotace (server.py:360-368)~~ ✅ OPRAVENO
|
|
Stejný SPK lze nahrát opakovaně. Žádný nonce/timestamp v podpisu. SPK se nikdy nerotuje → kompromitovaný SPK = trvalé dešifrování nových sessions.
|
|
**Opraveno:** SPK rotace každých 7 dní (`SPK_ROTATION_DAYS`). Server vrací `spk_created_at` v `handle_get_prekey_count`. `_ensure_prekeys()` kontroluje stáří SPK a rotuje pokud >= 7 dní. Předchozí SPK uložen jako grace period (`prev_spk_private.bin`/`prev_spk_id.txt`) pro in-flight X3DH. `_process_x3dh_header` přijímá `spk_override`, `_decrypt_dm` retry s předchozím SPK při selhání dešifrování.
|
|
|
|
#### ~~M5. Rate limit unbounded memory (server.py:73-83)~~ ✅ OPRAVENO (součást H4 fixu)
|
|
Staré záznamy se nikdy nečistí pokud klíč přestane být aktivní → útočník vytvoří miliony klíčů → memory leak.
|
|
**Opraveno:** `_cleanup_rate_limits()` v periodic cleanup (každých 10 min) maže stale entries z `rate_limits` i `connection_counts`.
|
|
|
|
#### ~~M6. TOCTOU race v membership checks (db.py)~~ ✅ OPRAVENO
|
|
`is_conversation_member()` → `remove_conversation_member()` — mezi kontrolou a akcí může jiný klient stav změnit.
|
|
**Opraveno:** `db.remove_conversation_member_atomic()` vrací bool (True pokud řádek existoval). Použito v `handle_remove_member` a `handle_leave_group`.
|
|
|
|
#### M7. MySQL spojení bez TLS (db.py:20-28)
|
|
`get_connection()` nepředává SSL parametry. Na vzdáleném serveru jdou credentials v plaintextu.
|
|
|
|
#### ~~M8. Chybějící validace UUID formátu (server.py: throughout)~~ ✅ OPRAVENO
|
|
`conv_id`, `user_id` — kontrola jen na neprázdnost, ne na formát UUID v4.
|
|
**Opraveno:** `_valid_file_id()` přejmenováno na `_valid_uuid()`. UUID validace přidána do všech handlerů přijímajících klientem poskytnuté `conv_id`, `user_id`, `message_id`, `device_id`.
|
|
|
|
#### ~~M9. Ratchet state corruption recovery (chat_core.py:1088-1104)~~ ✅ OPRAVENO
|
|
Pokud `decrypt()` změní chain keys ale selže na AAD verification, backup/restore mechanismus funguje, ale pokud backup selže (out-of-memory), stav zůstane corrupted.
|
|
**Opraveno:** `DoubleRatchet.decrypt()` nyní snapshotuje stav přes `_snapshot()`/`_restore()` a rollbackuje při jakékoliv výjimce (včetně skipped message key restore). `SenderKeyState.decrypt()` stejně snapshotuje `chain_key`, `n`, `_known_keys` před fast-forward a rollbackuje při selhání.
|
|
|
|
#### ~~M10. Chybějící validace velikosti message_ids listu (db.py:641-646)~~ ✅ OPRAVENO
|
|
Klient může poslat tisíce message_ids v jednom požadavku → pomalý SQL dotaz, DoS.
|
|
**Opraveno:** `handle_mark_read` nyní odmítá požadavky s více než 500 message_ids.
|
|
|
|
### 🟢 LOW — Dobrá praxe, nízké riziko
|
|
|
|
- **L1.** Hex string keys v skipped messages dict — timing side-channel po úspěšné autentikaci (crypto_utils.py:425)
|
|
- **L2.** RatchetHeader serializace redundantně konvertuje typy (crypto_utils.py:394-405)
|
|
- **L3.** `notif_label.setText()` bezpečné proti XSS (Qt neinterpretuje HTML v setText), ale křehké — přepnutí na `setHtml()` by to rozbilo (gui_client.py:1524, 2259)
|
|
- **L4.** SQL column interpolation v `update_user_profile` — whitelist chrání, ale pattern je nebezpečný při kopírování (db.py:818-822)
|
|
- **L5.** Chybějící TLS cipher suite hardening — Python defaulty jsou rozumné, ale ne explicitně nastavené (protocol.py)
|
|
- **L6.** Temporary pairing key není bezpečně vymazán z paměti (chat_core.py:581)
|
|
- **L7.** `_user_cache` ukládá public identity keys indefinitely — memory leak pro hodně kontaktů
|
|
|
|
### Druhý bezpečnostní review (zaměření na návrh, DB, komunikaci, lokální tmp/cache)
|
|
|
|
#### C6. Path traversal → libovolný zápis/smazání souborů přes file_id (server.py)
|
|
`handle_upload_image_start` vytváří soubor `UPLOAD_DIR / f"{file_id}.tmp"` bez validace `file_id`. Útočník může poslat `../../...` a zapisovat mimo UPLOAD_DIR. Následné rename, cleanup, `delete_message` a `delete_conversation` pak mohou mazat libovolné soubory.
|
|
**Řešení:** Striktně validovat file_id (UUID hex/kanonický formát), odmítnout cokoliv s `/`, `\`, `..`. Ověřit `path.resolve().is_relative_to(UPLOAD_DIR.resolve())`. Ideálně ukládat do podadresářů podle hash/UUID.
|
|
|
|
#### ~~H12. OPK race condition — reuse one-time pre-keys (db.py)~~ ✅ OPRAVENO
|
|
V `db.get_key_bundle()` se OPK vybírá SELECT → DELETE bez transakčního zámku. Při souběhu může být stejný OPK vydán vícekrát → porušení bezpečnostních předpokladů X3DH.
|
|
**Opraveno:** `consume_one_time_prekey()` a `get_key_bundle()` nyní používají `SELECT ... FOR UPDATE` + DELETE v jedné transakci (součást multi-device implementace).
|
|
|
|
#### H13. Neautentizované get_user_info + identity key exfiltrace (server.py)
|
|
`get_user_info` je dostupné bez přihlášení a vrací username, email a identity_key pro libovolný email/user_id. Umožňuje enumeraci uživatelů a sběr metadata/klíčů.
|
|
**Řešení:** Vyžadovat auth, nebo omezit na "kontakty v konverzaci".
|
|
|
|
#### ~~H14. TLS_INSECURE umožňuje MITM i v produkci (chat_core.py, server.py)~~ ✅ OPRAVENO
|
|
`TLS_INSECURE=true` vypíná verifikaci certifikátu → útočník může podvrhnout key bundle. Přímo ohrožuje E2EE integritu.
|
|
**Opraveno:** `TLS_INSECURE` vyžaduje `ENVIRONMENT=dev`, jinak RuntimeError. Součást H2 fixu.
|
|
|
|
#### ~~M11. Pairing poll DoS — neautentizovaný přístup k payload (server.py)~~ ✅ OPRAVENO
|
|
Kdokoli s 8-místným kódem může pollovat a "vyzvednout" payload (smazán po vyzvednutí). I když je šifrovaný, jde o snadný DoS (reálnému zařízení pairing selže).
|
|
**Opraveno:** `handle_pairing_start` generuje `poll_token` (secrets.token_hex(16)), vrací klientovi. `handle_pairing_poll` vyžaduje `poll_token` a porovnává přes `secrets.compare_digest()`. Klient ukládá token v `pairing_start()` a posílá v `pairing_wait()`.
|
|
|
|
#### ~~M12. Upload end bez validace received_bytes == file_size (server.py)~~ ✅ OPRAVENO
|
|
`upload_image_end` neověřuje, že `received_bytes == file_size`. Může zůstat nedokončený/nevalidní soubor.
|
|
**Řešení:** Kontrola délky před `complete_image_upload`.
|
|
|
|
#### M13. Klíčové adresáře bez chmod 700 (chat_core.py)
|
|
`get_key_dir` a podadresáře (`sessions`, `sender_keys`, `opk_private`) se vytvářejí bez explicitních práv; spoléhá se na umask.
|
|
**Řešení:** Po `mkdir` vždy `chmod 0o700` pro adresář, `0o600` pro soubory.
|
|
|
|
#### ~~L8. Phantom users — DB inflation (server.py, db.py)~~ ✅ OPRAVENO
|
|
`find_conversation` vytváří phantom usery pro libovolné emaily. I s rate limit lze DB časem nafouknout.
|
|
**Opraveno:** `_valid_email()` validace před vytvořením phantomu. `db.cleanup_stale_phantoms(30)` v periodic cleanup — maže phantomy starší 30 dní bez aktivních konverzací s reálnými uživateli.
|
|
|
|
### Bezpečnostní matice (souhrn)
|
|
|
|
| Soubor | CRITICAL | HIGH | MEDIUM | LOW |
|
|
|--------|----------|------|--------|-----|
|
|
| `protocol.py` | 1 (C1) | 2 (H5, H6) | 0 | 1 (L5) |
|
|
| `crypto_utils.py` | 1 (C2) | 0 | 3 (M1-M3) | 2 (L1, L2) |
|
|
| `server.py` | 2 (C5, C6) | 3 (H3, H7, H13) | 4 (M4, M8, M11, M12) | 0 |
|
|
| `chat_core.py` | 2 (C3, C4) | 4 (H1, H2/H14, H8, H9) | 2 (M9, M13) | 1 (L6) |
|
|
| `gui_client.py` | 0 | 2 (H10, H11) | 0 | 2 (L3, L7) |
|
|
| `db.py` | 0 | 1 (H12) | 3 (M6, M7, M10) | 2 (L4, L8) |
|
|
| **Celkem** | **6** | **12** | **12** | **8** |
|
|
| **Opraveno** | 6 (~~C1~~, ~~C2~~, ~~C3~~, ~~C4~~, ~~C5~~, ~~C6~~) | 11 (~~H1~~, ~~H2~~, ~~H3~~, ~~H4~~, ~~H5~~, ~~H6~~, ~~H7~~, ~~H8~~, ~~H10~~, ~~H11~~, ~~H12~~, ~~H13~~, ~~H14~~) | 11 (~~M2~~, ~~M3~~, ~~M4~~, ~~M5~~, ~~M6~~, ~~M8~~, ~~M9~~, ~~M10~~, ~~M11~~, ~~M12~~, ~~M13~~) | 1 (~~L8~~) |
|
|
| **Zbývá** | **0** | **1** | **1** | **7** |
|
|
|
|
### Doporučené pořadí oprav (aktualizováno)
|
|
1. ~~**C6**~~ ✅ — Path traversal přes file_id — DONE (UUID validace + is_relative_to)
|
|
2. ~~**C1 + C2 + C5**~~ ✅ — DoS vektory — DONE (LimitOverrunError → ValueError, MAX_SENDER_KEY_SKIP 256, upload completeness check)
|
|
3. ~~**H12**~~ ✅ — OPK race condition — DONE (SELECT FOR UPDATE, součást multi-device)
|
|
4. ~~**C3 + H1 + H7 + M13**~~ ✅ — Šifrování dat na disku + file/dir permissions + avatar path traversal — DONE
|
|
5. ~~**H2/H14**~~ ✅ — TLS hardening (TLS_INSECURE + TLS_AUTOGEN vyžadují ENVIRONMENT=dev, warning log) — DONE
|
|
6. ~~**H5 + H6**~~ ✅ — Error handling v protokolu (base64, JSON) — DONE
|
|
7. ~~**H3 + H13**~~ ✅ — User enumeration (generické odpovědi, auth pro get_user_info) — DONE
|
|
8. ~~**H4**~~ ✅ — Race conditions (asyncio.Lock) — DONE
|
|
9. ~~**H8 + H10 + H11**~~ ✅ — Paměť hesel, image parsing, filename sanitizace — DONE
|
|
10. ~~**M2 + M8 + M10 + M11**~~ ✅ — Hardening batch (HKDF salt, UUID validace, message_ids cap, pairing poll token) — DONE
|
|
11. ~~**M3, M4, M9**~~ ✅ — PBKDF2 600k iterations, SPK rotace 7 dní s grace period, ratchet state rollback — DONE
|
|
12. **M1, ~~M6~~, M7** — Remaining hardening (Ed25519 serialization, ~~TOCTOU~~ ✅, MySQL TLS)
|
|
13. **Penetrační testy** — manuální + automatizované security testy
|
|
|
|
## Důležitá rozhodnutí a kontext
|
|
- **Invitations replace direct add for groups:** `handle_add_member` and `handle_create_conversation` (for groups) now create invitations instead of directly adding members. DMs still auto-add both users. This was a design decision to give users control over joining groups.
|
|
- **Group delete = total destruction:** When creator deletes a group, ALL members are removed and the conversation is fully deleted. This is different from "leave group" which only removes the leaving user.
|
|
- **DM delete is per-user:** Deleting a DM only removes the deleting user. The other user still sees the conversation until they also delete it.
|
|
- **Avatar caching in GUI is pixmap-based:** `_avatar_cache` and `_group_avatar_cache` store QPixmap objects, not raw bytes. The `_on_avatar_for_conv_list` and `_on_group_avatar_for_conv_list` signals convert bytes → QImage → QPixmap on receipt.
|
|
- **No context menu on conversation list anymore:** Delete was the only action. Now handled by header buttons. `conv_list.setContextMenuPolicy(DefaultContextMenu)`.
|
|
|
|
## Environment Variables
|
|
See README.md for full list. Key: `SERVER_HOST`, `SERVER_PORT`, `MYSQL_*`, `TLS_*`, `SMTP_*`, `LOG_LEVEL`, `MAX_INPUT_CHARS`, `UPLOAD_DIR`, `MAX_IMAGE_BYTES`, `MAX_FILE_BYTES`, `MAX_MESSAGE_BYTES`.
|
|
|
|
## Commands & Workflow
|
|
|
|
- Start server: `python server.py`
|
|
- Start GUI client: `python gui_client.py`
|
|
- Start CLI client: `python client.py`
|
|
- Environment: `.env` file in project root (loaded by `dotenv`)
|
|
- Dependencies: `PyQt6`, `mysql-connector-python`, `cryptography`, `Pillow` (for image sharing), `python-dotenv`
|
|
- Check syntax: `python3 -m py_compile <file>.py`
|
|
- All files on both server AND client side: `crypto_utils.py`, `protocol.py`, `chat_core.py`, `gui_client.py` (or `client.py`)
|