Implement new style cookies

docs/README.md

@@ -2157,6 +2157,85 @@ $ http --session-read-only=./ro-session.json pie.dev/headers Custom-Header:orig-
 $ http --session-read-only=./ro-session.json pie.dev/headers Custom-Header:new-value
 ```
 
+### Host-based Cookie Policy
+
+Cookies in stored HTTPie sessions have a `domain` field which is binding them to the
+specified hostname. For example, in the following session:
+
+```json
+{
+    "cookies": [
+        {
+            "domain": "pie.dev",
+            "name": "secret_cookie",
+            "value": "value_1"
+        },
+        {
+            "domain": "httpbin.org",
+            "name": "secret_cookie",
+            "value": "value_2"
+        }
+    ]
+}
+```
+
+we will send `Cookie:secret_cookie=value_1` only when you are making a request against `pie.dev` (it
+also includes the domains, like `api.pie.dev`), and `Cookie:secret_cookie=value_2` when you use `httpbin.org`.
+
+```bash
+$ http --session=./session.json pie.dev/cookies
+```
+
+```json
+{
+    "cookies": {
+        "secret_cookie": "value_1"
+    }
+}
+```
+
+```bash
+$ http --session=./session.json httpbin.org/cookies
+```
+
+```json
+{
+    "cookies": {
+        "secret_cookie": "value_2"
+    }
+}
+```
+
+If you want to make a cookie domain unbound, you can simply set the `domain`
+field to `null` by editing the session file directly:
+
+```json
+{
+    "cookies": [
+        {
+            "domain": null,
+            "expires": null,
+            "name": "generic_cookie",
+            "path": "/",
+            "secure": false,
+            "value": "generic_value"
+        }
+    ]
+}
+```
+
+```bash
+$ http --session=./session.json pie.dev/cookies
+```
+
+```json
+{
+    "cookies": {
+        "generic_cookie": "generic_value"
+    }
+}
+```
+
 ### Cookie Storage Behavior
 
 **TL;DR:** Cookie storage priority: Server response > Command line request > Session file
@@ -2208,6 +2287,50 @@ Expired cookies are never stored.
 If a cookie in a session file expires, it will be removed before sending a new request.
 If the server expires an existing cookie, it will also be removed from the session file.
 
+### Upgrading Sessions
+
+In rare circumstances, HTTPie makes changes in it's session layout. For allowing a smoother transition of existing files
+from the old layout to the new layout we offer 2 interfaces:
+
+- `httpie cli sessions upgrade`
+- `httpie cli sessions upgrade-all`
+
+
+With `httpie cli sessions upgrade`, you can upgrade a single session with it's name (or it's path, if it is an
+[anonymous session](#anonymous-sessions)) and the hostname it belongs to. For example:
+
+([named session](#named-sessions))
+
+```bash
+$ httpie cli sessions upgrade pie.dev api_auth
+Refactored 'api_auth' (for 'pie.dev') to the version 3.1.0.
+```
+
+([anonymous session](#anonymous-sessions))
+
+```bash
+$ httpie cli sessions upgrade pie.dev ./session.json
+Refactored 'session' (for 'pie.dev') to the version 3.1.0.
+```
+
+If you want to upgrade every existing [named session](#named-sessions), you can use `httpie cli sessions upgrade-all` (be aware
+that this won't upgrade [anonymous sessions](#anonymous-sessions)):
+
+```bash
+$ httpie cli sessions upgrade-all
+Refactored 'api_auth' (for 'pie.dev') to the version 3.1.0.
+Refactored 'login_cookies' (for 'httpie.io') to the version 3.1.0.
+```
+
+#### Additional Customizations
+
+| Flag             | Description                                                                                                                                                                                                                  |
+|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `--bind-cookies` | Bind all the unbound cookies to the hostname that session belongs. By default, if the cookie is unbound (the `domain` attribute does not exist / set to an empty string) then it will still continue to be a generic cookie. |
+
+These flags can be used to customize the defaults during an `upgrade` operation. They can
+be used in both `sessions upgrade` and `sessions upgrade-all`.
+
 ## Config
 
 HTTPie uses a simple `config.json` file.
@@ -2299,6 +2422,23 @@ And since there’s neither data nor `EOF`, it will get stuck. So unless you’r
 
 Also, it might be good to set a connection `--timeout` limit to prevent your program from hanging if the server never responds.
 
+### Security
+
+#### Exposure of Cookies To The 3rd Party Hosts On Redirects
+
+*Vulnerability Type*: [CWE-200](https://cwe.mitre.org/data/definitions/200.html)
+*Severity Level*: LOW
+*Affected Versions*: `<3.1.0`
+
+The handling of [cookies](#cookies) was not compatible with the [RFC 6265](https://datatracker.ietf.org/doc/html/rfc6265)
+on the point of handling the `Domain` attribute when they were saved into [session](#sessions) files. All cookies were shared
+across all hosts during the runtime, including redirects to the 3rd party hosts.
+
+This vulnerability has been fixed in [3.1.0](https://github.com/httpie/httpie/releases/tag/3.1.0) and the
+[`httpie cli sessions upgrade`](#upgrading-sessions)/[`httpie cli sessions upgrade-all`]((#upgrading-sessions) commands
+have been put in place in order to allow a smooth transition to the new session layout from the existing [session](#sessions)
+files.
+
 ## Plugin manager
 
 HTTPie offers extensibility through a [plugin API](https://github.com/httpie/httpie/blob/master/httpie/plugins/base.py),

httpie/client.py

@@ -44,6 +44,7 @@ def collect_messages(
     httpie_session_headers = None
     if args.session or args.session_read_only:
         httpie_session = get_httpie_session(
+            env=env,
             config_dir=env.config.directory,
             session_name=args.session or args.session_read_only,
             host=args.headers.get('Host'),
@@ -130,10 +131,7 @@ def collect_messages(
     if httpie_session:
         if httpie_session.is_new() or not args.session_read_only:
             httpie_session.cookies = requests_session.cookies
-            httpie_session.remove_cookies(
-                # TODO: take path & domain into account?
-                cookie['name'] for cookie in expired_cookies
-            )
+            httpie_session.remove_cookies(expired_cookies)
             httpie_session.save()
 
 

httpie/config.py

@@ -1,7 +1,7 @@
 import json
 import os
 from pathlib import Path
-from typing import Union
+from typing import Any, Dict, Union
 
 from . import __version__
 from .compat import is_windows
@@ -62,6 +62,21 @@ class ConfigFileError(Exception):
     pass
 
 
+def read_raw_config(config_type: str, path: Path) -> Dict[str, Any]:
+    try:
+        with path.open(encoding=UTF8) as f:
+            try:
+                return json.load(f)
+            except ValueError as e:
+                raise ConfigFileError(
+                    f'invalid {config_type} file: {e} [{path}]'
+                )
+    except FileNotFoundError:
+        pass
+    except OSError as e:
+        raise ConfigFileError(f'cannot read {config_type} file: {e}')
+
+
 class BaseConfigDict(dict):
     name = None
     helpurl = None
@@ -77,26 +92,25 @@ def ensure_directory(self):
     def is_new(self) -> bool:
         return not self.path.exists()
 
+    def pre_process_data(self, data: Dict[str, Any]) -> Dict[str, Any]:
+        """Hook for processing the incoming config data."""
+        return data
+
+    def post_process_data(self, data: Dict[str, Any]) -> Dict[str, Any]:
+        """Hook for processing the outgoing config data."""
+        return data
+
     def load(self):
         config_type = type(self).__name__.lower()
-        try:
-            with self.path.open(encoding=UTF8) as f:
-                try:
-                    data = json.load(f)
-                except ValueError as e:
-                    raise ConfigFileError(
-                        f'invalid {config_type} file: {e} [{self.path}]'
-                    )
-                self.update(data)
-        except FileNotFoundError:
-            pass
-        except OSError as e:
-            raise ConfigFileError(f'cannot read {config_type} file: {e}')
-
-    def save(self):
-        self['__meta__'] = {
-            'httpie': __version__
-        }
+        data = read_raw_config(config_type, self.path)
+        if data is not None:
+            data = self.pre_process_data(data)
+            self.update(data)
+
+    def save(self, *, bump_version: bool = False):
+        self.setdefault('__meta__', {})
+        if bump_version or 'httpie' not in self['__meta__']:
+            self['__meta__']['httpie'] = __version__
         if self.helpurl:
             self['__meta__']['help'] = self.helpurl
 
@@ -106,13 +120,19 @@ def save(self):
         self.ensure_directory()
 
         json_string = json.dumps(
-            obj=self,
+            obj=self.post_process_data(self),
             indent=4,
             sort_keys=True,
             ensure_ascii=True,
         )
         self.path.write_text(json_string + '\n', encoding=UTF8)
 
+    @property
+    def version(self):
+        return self.get(
+            '__meta__', {}
+        ).get('httpie', __version__)
+
 
 class Config(BaseConfigDict):
     FILENAME = 'config.json'

httpie/manager/cli.py

@@ -2,6 +2,15 @@
 from httpie.cli.argparser import HTTPieManagerArgumentParser
 from httpie import __version__
 
+CLI_SESSION_UPGRADE_FLAGS = [
+    {
+        'variadic': ['--bind-cookies'],
+        'action': 'store_true',
+        'default': False,
+        'help': 'Bind domainless cookies to the host that session belongs.'
+    }
+]
+
 COMMANDS = {
     'plugins': {
         'help': 'Manage HTTPie plugins.',
@@ -34,6 +43,34 @@
             'List all installed HTTPie plugins.'
         ],
     },
+    'cli': {
+        'help': 'Manage HTTPie for Terminal',
+        'sessions': {
+            'help': 'Manage HTTPie sessions',
+            'upgrade': [
+                'Upgrade the given HTTPie session with the latest '
+                'layout. A list of changes between different session versions '
+                'can be found in the official documentation.',
+                {
+                    'dest': 'hostname',
+                    'metavar': 'HOSTNAME',
+                    'help': 'The host this session belongs.'
+                },
+                {
+                    'dest': 'session',
+                    'metavar': 'SESSION_NAME_OR_PATH',
+                    'help': 'The name or the path for the session that will be upgraded.'
+                },
+                *CLI_SESSION_UPGRADE_FLAGS
+            ],
+            'upgrade-all': [
+                'Upgrade all named sessions with the latest layout. A list of '
+                'changes between different session versions can be found in the official '
+                'documentation.',
+                *CLI_SESSION_UPGRADE_FLAGS
+            ],
+        }
+    }
 }
 
 
@@ -54,6 +91,8 @@ def generate_subparsers(root, parent_parser, definitions):
     )
     for command, properties in definitions.items():
         is_subparser = isinstance(properties, dict)
+        properties = properties.copy()
+
         descr = properties.pop('help', None) if is_subparser else properties.pop(0)
         command_parser = actions.add_parser(command, description=descr)
         command_parser.root = root
@@ -62,7 +101,9 @@ def generate_subparsers(root, parent_parser, definitions):
             continue
 
         for argument in properties:
-            command_parser.add_argument(**argument)
+            argument = argument.copy()
+            variadic = argument.pop('variadic', [])
+            command_parser.add_argument(*variadic, **argument)
 
 
 parser = HTTPieManagerArgumentParser(

httpie/manager/core.py

@@ -1,9 +1,11 @@
 import argparse
+from typing import Optional
 
 from httpie.context import Environment
 from httpie.manager.plugins import PluginInstaller
 from httpie.status import ExitStatus
 from httpie.manager.cli import missing_subcommand, parser
+from httpie.manager.tasks import CLI_TASKS
 
 MSG_COMMAND_CONFUSION = '''\
 This command is only for managing HTTPie plugins.
@@ -22,12 +24,21 @@
 '''.rstrip("\n").format(args='POST pie.dev/post hello=world')
 
 
+def dispatch_cli_task(env: Environment, action: Optional[str], args: argparse.Namespace) -> ExitStatus:
+    if action is None:
+        parser.error(missing_subcommand('cli'))
+
+    return CLI_TASKS[action](env, args)
+
+
 def program(args: argparse.Namespace, env: Environment) -> ExitStatus:
     if args.action is None:
         parser.error(MSG_NAKED_INVOCATION)
 
     if args.action == 'plugins':
         plugins = PluginInstaller(env, debug=args.debug)
         return plugins.run(args.plugins_action, args)
+    elif args.action == 'cli':
+        return dispatch_cli_task(env, args.cli_action, args)
 
     return ExitStatus.SUCCESS