[HttpFoundation] Added structured namespacing to session attributes. #2541

ghost · 2011-11-02T09:09:33Z

Bug fix: no
Feature addition: yes
Backwards compatibility break: no
Symfony2 tests pass: yes
Fixes the following tickets: -

Symfony's session class works well for simple applications, for complex systems with multiple components/modules all working together (think Drupal, Zikula etc), then there can easily be session attribute clashes.

This patch introduces structured namespacing which allows deep storage according to the namespace. Namespaces are expressed using paths, e.g. /users, or /users/permissions and this translates into a structured array of data, which is obviously returned when using get() and the namespace.

I have also taken the opportunity to complete the docblocks for the session class' properties which greatly assists programming when using an IDE.

RATIONALE FOR STRUCTURED NAMESPACING

Here are some examples to see why this 'structured namespace' methodology is extremely convenient over using a flat system. Let's look at an example with csrf tokens. Let's say we have multiple csrftokens stored by form ID (allowing multiple forms on the page and tabbed browsing).

If we're using a flat system, you might have

'tokens' => array('a' => 'a6c1e0b6',
                  'b' => 'f4a7b1f3')

With a flat system when you get the key tokens, you will get back an array, so now you have to analyse the array. So if you simply want to add another token, you have to follow three steps: get the session attribute tokens, have to add to the array, and lastly set the entire array back to the session.

$tokens = $session->get('tokens');
$tokens['c'] = $value;
$session->set('tokens', $tokens);

Doable, but you can see it's pretty long winded.

With structured namespacing you can simply do:

$session->set('c', $value, '/tokens');

henrikbjorn · 2011-11-02T09:11:52Z

what is it with all the & by reference stuff ?

stof · 2011-11-02T09:12:22Z

src/Symfony/Component/HttpFoundation/Session.php

    {
-        return array_key_exists($name, $this->attributes);
+        $attributes = & $this->resolveAttributePath($namespace);


why = & ?

This is functional and not an optimization. We are returning a reference to a key inside the $this->attributes property. This allows you to get structured data rather than a flat array . this allowing deep namespacing,

I don't see why you need a reference when doing an array_key_exists call on it.

We are not addressing flat arrays, ie $array[$key], but potentially deep arrays, like $array['user']['permissions'].

this does not change anything to the fact that the method does not modify the array at all so assigning it by reference is useless

I know it might look odd, but it's required, if you remove it the code stops working. It's functional, not optimizational (sic). Remember, we're working on $this->attributes ultimately, or somewhere inside the structured array that is $this->attributes.

I don't see how a read-only check could stop working if you use it with a real array instead of a reference. I don't ask removing the use of references everywhere, only here.

@stof - ah, I misunderstood - let me check the consequences and update the PR as necessary.

to be exact, the get method is also concerned as it is also read-only.

@stof - I've removed the reference from get() and has() methods in a01e624

The use of references in this PR are entirely functional and required for set() methods, however in the case of reading, they are not necessary. They are probably not doing any harm, but the rule of thumb in PHP 5 is to only use references for functionality reasons.

fabpot · 2011-11-02T13:29:48Z

I don't see a need for namespaces as the keys can be anything you want, including a path-like name.

ghost · 2011-11-02T16:49:45Z

Imagine the convenience of being able to operate on groups of session variables, or on single variables. If namespacing is simply flat, e.g.

When it's flat you can't remove a whole bunch of attributes in a hierarchy. Say we have a module 'foo', then we

foo.bar = 123
foo.lux = array(1,2,3)
foo.tokens.a = value
foo.tokens.b = value

Now if this was stored in a hierarchy (and the programmer doesn't need to be concerned with how it's stored) we have

/foo - bar = 123
/foo - lux = array(1,2,3)
/foo - tokens/a = value
/foo - tokens/b = value

We can use $session->remove('tokens', '/foo') for example to remove all the tokens. If this was stored as flat we simply couldn't do that.

This is just one example to look at. There is a lot more flexibility doing it like this.

rmburkhead · 2011-11-02T16:59:07Z

Drak has added two things here: namespacing, and a hierarchical structure to the namespace path.

With respect to just namespacing and your comment about key naming, I agree to an extent. It is enough to provide a unique-enough name to the key to avoid variable name collisions. Namespacing, however, promotes the concept of name differentiation to a first-class concept, instead of something the diligent programmer needs to remember to do. One thing a well-written API sometimes does is promote a concept to the user of that API in order to ensure that some convention is followed or to prompt the user of the API to remember a requirement of using it. Adding namespacing does this for the concept of variable name differentiation.

Adding a hierarchical structure to the concept of namespacing adds convenience for the programmer. You mentioned that keys can be "path-like." Path-like names, however, are meaningless unless there is some support for the pathing aspect of the naming convention. Drak has codified that. In terms of convenience, this allows the programmer to, for example, add the variables 'settings' and 'breadcrumbs' to the namespace '/myapplication', and then to reset them one could simply reset the variable 'myapplication' in the top-level '/' namespace. These types of aggregate operations only become possible when a hierarchical structure is applied to the naming of variables, or in this case, applied to the namespacing.

henrikbjorn · 2011-11-02T19:00:03Z

Could this fix the problem with session flashes not being able to have a translation domain?. Where a method can get all sessions and their domains so you can do {{ flash|trans({}, flashDomain) }} ?

fabpot · 2011-11-03T09:00:35Z

I totally understand the namespace concept as it is available in symfony1. And from my experience, this was rarely used. That's why support for it has not been added in Symfony2.

ghost · 2011-11-03T11:19:28Z

@henrikbjorn -This PR is focused on session attributes, not flash messages. But in any case, in my experience with flash messages in other ML systems, there is no particular need to have locale handling in the flash messages API itself. This is because any given user session is already in a given language, so when a flash message is being set, it would be translated and the translated flash stored, e.g. (pseudocode)

$session->addFlash(translate('Username or password were invalid'));

When it comes to displaying the flash messages in a view template, you just get them all by type and render them accordingly. Or is there something I am missing?

ghost · 2011-11-03T16:26:54Z

@fabpot - For us at Zikula it is pretty useful. We're seriously trying to migrate as much of the framework stuff we need to Symfony2 so we can focus on what we specialise in rather than reinvent the wheel.

Systems like Zikula and Drupal are designed to build very complex systems with multiple modules/applications working together to create the functionality in a website and Symfony2 neatly deals with the underlying request/routing/response stuff. But if you want to see Symfony2 components use grow widely, they need to be suitable for complex systems like Drupal/Zikula. While sf components are amazing, they are sometimes lacking in essentials: take the lack of flash message typing is a good example - something the Drupal team have also mentioned and for which there is now a PR.

HttpFoundation is a pretty core component and it needs to be flexible enough for both simple and complex use right out of the box. It is possible to just create a naming convention for session attributes, that works, but it doesn't give the other benefits of having hierarchy listed above.

ghost · 2011-11-10T02:59:00Z

I've got some more examples of why this 'structured namespace' methodology is extremely convenient over using a flat system. I'll go with the tokens example again. Let's say we have multiple csrftokens stored by form ID (allowing multiple forms on the page and tabbed browsing).

If we're using a flat system, you might have

'tokens' => array('a' => 'a6c1e0b6',
                  'b' => 'f4a7b1f3')

With a flat system when you get the key tokens, you will get back an array, so now you have to analyse the array. So if you simply want to add another token, you have to follow three steps: get the session attribute tokens, have to add to the array, and lastly set the entire array back to the session.

$tokens = $session->get('tokens');
$tokens['c'] = $value;
$session->set('tokens', $tokens);

Doable, but you can see it's pretty long winded.

With structured namespacing you can simply do:

$session->set('c', $value, '/tokens');

ghost · 2011-11-16T05:44:16Z

FYI - I'm going AFK for about 5 or so days, I will try to work more on this offline and update you when I get back.

ghost · 2011-12-09T10:22:18Z

Closing this PR as changed incorporated in #2714

@deviantintegral

Commits ------- cb6fdb1 [HttpFoundation] removed Session::close() c59d880 Docblocks. b8df162 Correct instanceof condition. 8a01dd5 renamed getFlashes() to getFlashBag() to avoid clashes 282d3ae updated CHANGELOG for 2.1 0f6c50a [HttpFoundation] added some method for a better BC 146a502 [FrameworkBundle] added some service aliases to avoid some BC breaks 93d81a1 [HttpFoundation] removed configuration for session storages in session.xml as we cannot provide a way to configure them (like before this PR anyway) 74ccf70 reverted 5b7ef11 (Simplify session storage class names now we have a separate namespace for sessions) 91f4f8a [HttpFoundation] changed default flash bag to auto-expires to keep BC 0494250 removed unused use statements 7878a0a [HttpFoundation] renamed pop() to all() and getAll() to all() 0d2745f [HttpFoundation] Remove constants from FlashBagInterface dad60ef [HttpFoundation] Add back get defaults and small clean-up. 5b7ef11 [HttpFoundation] Simplify session storage class names now we have a separate namespace for sessions. 27530cb [HttpFoundation] Moved session related classes to own sub-namespace. 4683915 [HttpFoundation] Free bags from session storage and move classes to their own namespaces. d64939a [DoctrineBridge] Refactored driver for changed interface. f9951a3 Fixed formatting. 398acc9 [HttpFoundation] Reworked flashes to maintain same behaviour as in Symfony 2.0 f98f9ae [HttpFoundation] Refactor for DRY code. 9dd4dbe Documentation, changelogs and coding standards. 1ed6ee3 [DoctribeBridge][SecurityBundle][WebProfiler] Refactor code for HttpFoundation changes. 7aaf024 [FrameworkBundle] Refactored code for changes to HttpFoundation component. 669bc96 [HttpFoundation] Added pure Memcache, Memcached and Null storage drivers. e185c8d [HttpFoundation] Refactored component for session workflow. 85b5c43 [HttpFoundation] Added drivers for PHP native session save handlers, files, sqlite, memcache and memcached. 57ef984 [HttpFoundation] Added unit and functional testing session storage objects. 3a263dc [HttpFoundation] Introduced session storage base class and interfaces. c969423 [HttpFoundation] Added FlashBagInterface and concrete implementation. 39288bc [HttpFoundation] Added AttributesInterface and AttributesBagInterface and concrete implementations. Discussion ---------- [2.1][HttpFoundation] Refactor session handling and flash messages Bug fix: yes Feature addition: yes Backwards compatibility break: yes Symfony2 tests pass: yes Fixes the following tickets: #2607, #2591, #2717, #2773 References the following tickets: #2592, #2543, #2541, #2510, #2714, #2684 Todo: - __Introduction__ This extensive PR is a refactor with minimal BC breaks of the `[HttpFoundation]` component's session management which fixes several issues in the current implementation. This PR includes all necessary changes to other bundles and components is documented in the `CHANGELOG-2.1` and `UPGRADING-2.1`. __Summary of Changes__ __Session:__ - Session object now implements `SessionInterface` __Attributes:__ - Attributes now handled by `AttributeBagInterface` - Added two AttributeBag implementations: `AttributeBag` replicates the current Symfony2 attributes behaviour, and the second, `NamespacedAttributeBag` introduces structured namespaced representation using '/' in the key. Both are BC. `FrameworkBundle` defaults to the old behaviour. __Flash messages:__ - Flash messages now handled by `FlashBagInterface` - Introduced `FlashBag` which changes the way flash messages expire, they now expire on use rather than automatically, useful for ESI. - Introduced `AutoExpireFlashBag` (default) which replicates the old automatic expiry behaviour of flash messages. __Session Storage:__ - Introduced a base object, `AbstractSessionStorage` for session storage drivers - Introduced a `SessionSaveHandlerInterface` when using custom session save handlers - Introduced a `NullSessionStorage` driver which allows for unsaved sessions - Introduced new session storage drivers for Memcache and Memcached - Introduced new session storage drivers for PHP's native SQLite, Memcache and Memcached support __General:__ - Fixed bugs where session attributes are not saved and all cases where flash messages would get lost - Wrote new tests, refactored related existing tests and increased test coverage extensively. __Rationale/Details__ I'll explain more detail in the following sections. __Unit Tests__ All unit and functional tests pass. __Note on Functional Testing__ I've introduced `MockFileSessionStorage` which replaces `FilesystemSessionStorage` to emulate a PHP session for functional testing. Essentially the same methodology of functional testing has been maintained but without interrupting the other session storage drivers interaction with real PHP sessions. The service is now called `session.storage.mock_file`. __Session Workflow__ PHP sessions follow a specific workflow which is not being followed by the current session management implementation and is responsible for some unpredictable bugs and behaviours. Basically, PHP session workflow is as follows: `open`, `read`, `write`, `close`. In between these can occur, `destroy` and `garbage collection`. These actions are handled by `session save handlers` and one is always registered in all cases. By default, the `files` save handler (internally to PHP) is registered by PHP at the start of code execution. PHP offers the possibility to change the save handler to another internal type, for example one provided by a PHP extension (think SQLite, Memcache etc), or you can register a user type and set your own handlers. However __in all cases__ PHP requires the handlers. The handlers are called when the following things occur: - `open` and `read` when `session_start()` or the session autostarts when PHP outputs some display - `destroy` when `session_regenerate_id(true)` is called - `write` and `close` when PHP shuts down or when `session_write_close()` is called - `garbage collection` is called randomly according to configurable probability The next very important aspect of this PR is that `$_SESSION` plays an important part in this workflow because the contents of the $_SESSION is populated __after__ the `read` handler returns the previously saved serialised session data. The `write` handler is sent the serialised `$_SESSION` contents for save. Please note the serialisation is a different format to `serialize()`. For this reason, any session implementation cannot get rid of using `$_SESSION`. I wrote more details on this issue [here](#2607 (comment)) In order to make writing session storage drivers simple, I created a light base class `AbstractSessionStorage` and the `SessionSaveHandlerInterface` which allows you to quickly write native and custom save handler drivers. __Flash Messages [BC BREAK]__ Flash messages currently allow representation of a single message per `$name`. Fabien designed the original system so that `$name` was equivalent to flash message type. The current PR changes the fact that Flash messages expire explicitly when retrieved for display to the user as opposed to immediately on the next page load. The last issue fixes potential cases when flash messages are lost due to an unexpected intervening page-load (an error for example). The API `get()` has a flag which allows you to override the `clear()` action. __Flash message translation__ This PR does not cover translation of flash messages because messages should be translated before calling the flash message API. This is because flash messages are used to present messages to the user after a specific action, and in any case, immediately on the next page load. Since we know the locale of the request in every case we can translate the message before storing. Secondly, translation is simply a string manipulation. Translation API calls should always have the raw untranslated string present because it allows for extraction of translation catalogs. For a complete answer see my answer [here](#2543 (comment)) __Session attribute and structured namespacing__ __This has been implemented without changing the current default behaviour__ but details are below for the alternative: Attributes are currently stored in a flat array which limits the potential of session attributes: Here are some examples to see why this 'structured namespace' methodology is extremely convenient over using a flat system. Let's look at an example with csrf tokens. Let's say we have multiple csrftokens stored by form ID (allowing multiple forms on the page and tabbed browsing). If we're using a flat system, you might have 'tokens' => array('a' => 'a6c1e0b6', 'b' => 'f4a7b1f3') With a flat system when you get the key `tokens`, you will get back an array, so now you have to analyse the array. So if you simply want to add another token, you have to follow three steps: get the session attribute `tokens`, have to add to the array, and lastly set the entire array back to the session. $tokens = $session->get('tokens'); $tokens['c'] = $value; $session->set('tokens', $tokens); Doable, but you can see it's pretty long winded. With structured namespacing you can simply do: $session->set('c', $value, '/tokens'); There are several ways to implement this, either with an additional `$namespace` argument, or by treating a character in the `$key` as a namespacer. `NamespacedAttributeBag` treats `/` as a namespacer so you can represent `user.tokens/a` for example. The namespace character is configurable in `NamespacedAttributeBag`. --------------------------------------------------------------------------- by marijn at 2011-12-18T15:43:17Z I haven't read the code yet but the description from this PR and your line of thought seem very well structured. Seems like a big +1 for me. --------------------------------------------------------------------------- by lsmith77 at 2011-12-19T16:01:19Z @deviantintegral could you look over this to see if it really addresses everything you wanted with PR #2510 ? --------------------------------------------------------------------------- by deviantintegral at 2011-12-24T20:12:03Z I've read through the documentation and upgrade notes, and I can't see anything that's obviously missing from #2510. Being able to support multiple flashes per type is the most important, and the API looks reasonable to me. Drupal does support supressing repeat messages, but that can easily be implemented in our code unless there's a compelling case for it to be a part of Symfony. I wonder if PHP memcache support is required in Symfony given the availability of memcached. I'm not familiar with how other parts of Symfony handle it, but there is often quite a bit of confusion between the two PHP extensions. It could be simpler to remove one, or add a bit of info describing or linking to why there are two nearly identical classes. Is it possible to make one class inherit from the other (memcached is a child of memcache)? --------------------------------------------------------------------------- by Fristi at 2011-12-24T20:29:46Z Interesting, maybe add: session events as I did with the current impl: https://github.com/Fristi/SessionBundle --------------------------------------------------------------------------- by drak at 2011-12-25T00:50:03Z @deviantintegral - I agree about the confusion between memcache and memcached but actually, it is necessary to support both because `memcached` is not available everywhere. For example on Debian Lenny and RHEL/CentOS 5, only memcache is available by default. This would preclude a massive amount of shared hosting environments. Also, it is not possible to inherit one from the other, they are completely different drivers. @Fristi - I also thought about the events, but they do not belong as part of the standalone component as this would create a coupling to the event dispatcher. The way you have done it, ie, in a bundle is the right way to achieve it. --------------------------------------------------------------------------- by matheo at 2011-12-25T01:12:00Z Impressive work, looks like a big improvement and deserves a big +1 --------------------------------------------------------------------------- by datiecher at 2011-12-26T11:57:12Z Took some time to grok all the changes in this PR but all in all it is a keeper. Specially the new flash message API, it's really nicer to work with it then the previous one. Nicely done @Drak! --------------------------------------------------------------------------- by lsmith77 at 2012-01-02T15:00:00Z @fabpot did you have time to review this yet? with all the work @Drak has done its important that he gets some feedback soon. its clear this PR breaks BC in ways we never wanted to allow. but i think this PR also clearly explains why its necessary none the less. --------------------------------------------------------------------------- by drak at 2012-01-02T15:41:53Z @fabpot - I have removed the WIP status from this PR now and rebased against the current master branch. --------------------------------------------------------------------------- by Tobion at 2012-01-07T07:13:38Z From what I read from the IRC chat logs, the main concern of @fabpot is whether we really need multiple flash messages per type. I'm in favor of this PR and just want to add one point to this discussion. At the moment you can add multiple flash messages of different type/category/identifier. For example you can specify one error message and one info message after an operation. I think most agree that this can be usefull. But then it makes semantically no sense that you currently cannot add 2 info messages. This approach feels a bit half-done. So I think this PR eliminates this paradox. --------------------------------------------------------------------------- by drak at 2012-01-07T09:11:07Z For reference there is also a discussion started by @lsmith77 on the mailing list at https://groups.google.com/forum/#!topic/symfony-devs/cy4wokD0mQI --------------------------------------------------------------------------- by dlsniper at 2012-01-07T16:02:15Z @Drak I could also add the next scenario that I currently have to live with, in addition to @lsmith77 ones. I had this issue while working on our shopping cart implementation for a customer where the customer wanted to show the unavailability of the items as different lines in the 'flash-error' section of the cart. We had to set an array as the 'flash message' in order to display that information. So in this case for example having the flash messages types as array would actually make more sense that sending an array to the flasher. Plus the the other issue we had was that we also wanted to add another error in the message but we had to do a check to see if the flash message is an array already or we need to make it an array. I think it's better not to impose a limit of this sort and let the users be able to handle every scenario, even if some are rare, rather that forcing users to overcome limitations such as these. I really hope this PR gets approved faster and thanks everyone for their hard work :) --------------------------------------------------------------------------- by Tobion at 2012-01-07T21:01:07Z @dlsniper I think you misinterpreted my point. --------------------------------------------------------------------------- by dlsniper at 2012-01-07T21:04:04Z @Tobion I'm sorry I did that, I'll edit the message asap. Seems no sleep in 26 hours can cause brain not to function as intended :) --------------------------------------------------------------------------- by lsmith77 at 2012-02-01T14:38:52Z FYI the drupal guys are liking this PR (including the flash changes): http://drupal.org/node/335411 --------------------------------------------------------------------------- by drak at 2012-02-01T14:51:33Z @lsmith77 Fabien asked me to remove the changes to the flash messages so that they are as before - i.e. only one flash per name/type /cc @fabpot --------------------------------------------------------------------------- by fabpot at 2012-02-01T14:58:23Z To be clear, I've asked to split this PR in two parts: * one about the session refactoring (which is non-controversial and should be merged ASAP) * this one with only the flash refactoring --------------------------------------------------------------------------- by drak at 2012-02-02T11:29:26Z @fabpot this is ready to be merged now. I will open a separate PR later today for the flash messages as a bucket. --------------------------------------------------------------------------- by fabpot at 2012-02-02T11:34:39Z I must have missed something, but I still see a lot of changes related to the flash messages. --------------------------------------------------------------------------- by drak at 2012-02-02T11:39:10Z When I spoke to you you said you wanted to make the commit with flash messages with one message per name/type rather than multiple. The old flash messages behaviour is 100% maintained in `AutoExpireFlashBag` which can be the default in the framework if you wish. The `FlashBag` implementation makes Symfony2 ESI compatible. --------------------------------------------------------------------------- by stof at 2012-02-02T11:47:38Z @Drak splitting into 2 PRs means you should not refactor the flash messages in this one but in the dedicated one. --------------------------------------------------------------------------- by drak at 2012-02-02T12:29:43Z @stof Yes. I discussed with Fabien over chat there are basically no changes to flashes in `FlashBag` and `AutoExpireFlashBag` maintains the exact behaviour as before. The FlashBag just introduces ESI compatible flashes. There is no way to refactor the sessions without moving the flash messages to their own bag. The next PR will propose the changes to flashes that allow multiple messages per name/type. I can size the PR down a little more removing the new storage drivers and so on to make the PR smaller but that's really as far as I can go. To be clear, while the API has changed a little for flashes, the behaviour is the same.

Added structured namespacing to session attributes.

df69589

stof reviewed Nov 2, 2011
View reviewed changes

andrewtch mentioned this pull request Nov 20, 2011

[HttpFoundation] Wildcard implementation for ParameterBag & Session #2684

Closed

ghost mentioned this pull request Nov 24, 2011

[HttpFoundation] Refactor session handling and flash messages #2714

Closed

ghost closed this Dec 9, 2011

ghost mentioned this pull request Dec 12, 2011

[2.1][HttpFoundation] Refactor session handling and flash messages #2853

Merged

cabello mentioned this pull request Jun 20, 2013

[Security] RoleSecurityIdentity checks for instances of RoleInterface to allow custom Role implementation #8313

Closed

hacfi mentioned this pull request Jul 24, 2013

[2.3] Session set by namespace character not work. #8555

Closed

This pull request was closed.

Uh oh!

[HttpFoundation] Added structured namespacing to session attributes. #2541

[HttpFoundation] Added structured namespacing to session attributes. #2541

Uh oh!

Conversation

ghost commented Nov 2, 2011

Uh oh!

henrikbjorn commented Nov 2, 2011

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fabpot commented Nov 2, 2011

Uh oh!

ghost commented Nov 2, 2011

Uh oh!

rmburkhead commented Nov 2, 2011

Uh oh!

henrikbjorn commented Nov 2, 2011

Uh oh!

fabpot commented Nov 3, 2011

Uh oh!

ghost commented Nov 3, 2011

Uh oh!

ghost commented Nov 3, 2011

Uh oh!

ghost commented Nov 10, 2011

Uh oh!

ghost commented Nov 16, 2011

Uh oh!

ghost commented Dec 9, 2011

Uh oh!

Uh oh!