RFC - Escaper Class (Extended Escaping for Zend\View)

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

RFC - Escaper Class (Extended Escaping for Zend\View)

Pádraic Brady
Hi all,

I've added an Escaper Class RFC to the list of things you are going to
read this week. I know you'll all read it and provide valuable
feedback because you like building secure web applications. You
probably also want to continue living which will quickly cease if you
don't read this. Final warning, I have a whole team of ninja assassins
ready to be dispatched ;).

http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class

The RFC proposes a new class designed to offer secure escaping for
Zend\View beyond basic htmlspecialchars() usage. Its stated goal is to
implement contextual escaping based on peer-reviewed rules that carry
considerable weight as a recommended practice. In other words, I'm
relying on the published OWASP ESAPI codecs for guidance (you can find
the Open Web Application Security Project (OWASP) at
https://www.owasp.org but I'm sure you already have it bookmarked!).
As security problems, particularly Cross Site Scripting, have
continued to plague PHP applications, this approach goes a lot further
than you might suspect and it's intended to remove the need to create
custom escaping code and offer an even more secure platform to utilise
in building your applications with Zend Framework 2.0.

Your feedback is, as always, extremely welcome whether by comment or a
reply email.

Best regards,
Paddy

--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Stewart Lord
Hi Paddy,

This is great. I wonder if we could get this into ZF1? We have
implemented escapeJs() and escapeAttr() helpers (which you are welcome
to see) and it would be great if there were official versions that were
potentially faster or more secure. In any case I could backport your ZF2
versions for my own use if they are better.

Our helpers rely on iconv instead of mbstring. It's been a while since
I've worked on it, so I can't recall why we made that decision. I will
say that our version does character-by-character iteration and profiling
indicates performance is not great.

Regarding your plan to put these under Zend\View\Escaper, did you
consider making these filters?

Regards,
Stew


On 12-02-29 1:48 PM, Pádraic Brady wrote:

> Hi all,
>
> I've added an Escaper Class RFC to the list of things you are going to
> read this week. I know you'll all read it and provide valuable
> feedback because you like building secure web applications. You
> probably also want to continue living which will quickly cease if you
> don't read this. Final warning, I have a whole team of ninja assassins
> ready to be dispatched ;).
>
> http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class
>
> The RFC proposes a new class designed to offer secure escaping for
> Zend\View beyond basic htmlspecialchars() usage. Its stated goal is to
> implement contextual escaping based on peer-reviewed rules that carry
> considerable weight as a recommended practice. In other words, I'm
> relying on the published OWASP ESAPI codecs for guidance (you can find
> the Open Web Application Security Project (OWASP) at
> https://www.owasp.org but I'm sure you already have it bookmarked!).
> As security problems, particularly Cross Site Scripting, have
> continued to plague PHP applications, this approach goes a lot further
> than you might suspect and it's intended to remove the need to create
> custom escaping code and offer an even more secure platform to utilise
> in building your applications with Zend Framework 2.0.
>
> Your feedback is, as always, extremely welcome whether by comment or a
> reply email.
>
> Best regards,
> Paddy
>
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Tomáš Fejfar
In reply to this post by Pádraic Brady
Great job summing it up. I have better understanding of some of the escaping's caveans just by reading this. Including this would be IMO a huge leap towards security. Because knowing that there are still ZF users who have trouble understanding relatively simple stuff like decorators - it's obvious that there is a huge portion of users who have enough knowledge in such complex area as XSS protection... 

On Wed, Feb 29, 2012 at 10:48 PM, Pádraic Brady <[hidden email]> wrote:
Hi all,

I've added an Escaper Class RFC to the list of things you are going to
read this week. I know you'll all read it and provide valuable
feedback because you like building secure web applications. You
probably also want to continue living which will quickly cease if you
don't read this. Final warning, I have a whole team of ninja assassins
ready to be dispatched ;).

http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class

The RFC proposes a new class designed to offer secure escaping for
Zend\View beyond basic htmlspecialchars() usage. Its stated goal is to
implement contextual escaping based on peer-reviewed rules that carry
considerable weight as a recommended practice. In other words, I'm
relying on the published OWASP ESAPI codecs for guidance (you can find
the Open Web Application Security Project (OWASP) at
https://www.owasp.org but I'm sure you already have it bookmarked!).
As security problems, particularly Cross Site Scripting, have
continued to plague PHP applications, this approach goes a lot further
than you might suspect and it's intended to remove the need to create
custom escaping code and offer an even more secure platform to utilise
in building your applications with Zend Framework 2.0.

Your feedback is, as always, extremely welcome whether by comment or a
reply email.

Best regards,
Paddy

--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team

Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Tomáš Fejfar
Who DON'T have... ;)


On Thu, Mar 1, 2012 at 12:04 AM, Tomáš Fejfar <[hidden email]> wrote:
Great job summing it up. I have better understanding of some of the escaping's caveans just by reading this. Including this would be IMO a huge leap towards security. Because knowing that there are still ZF users who have trouble understanding relatively simple stuff like decorators - it's obvious that there is a huge portion of users who have enough knowledge in such complex area as XSS protection... 


On Wed, Feb 29, 2012 at 10:48 PM, Pádraic Brady <[hidden email]> wrote:
Hi all,

I've added an Escaper Class RFC to the list of things you are going to
read this week. I know you'll all read it and provide valuable
feedback because you like building secure web applications. You
probably also want to continue living which will quickly cease if you
don't read this. Final warning, I have a whole team of ninja assassins
ready to be dispatched ;).

http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class

The RFC proposes a new class designed to offer secure escaping for
Zend\View beyond basic htmlspecialchars() usage. Its stated goal is to
implement contextual escaping based on peer-reviewed rules that carry
considerable weight as a recommended practice. In other words, I'm
relying on the published OWASP ESAPI codecs for guidance (you can find
the Open Web Application Security Project (OWASP) at
https://www.owasp.org but I'm sure you already have it bookmarked!).
As security problems, particularly Cross Site Scripting, have
continued to plague PHP applications, this approach goes a lot further
than you might suspect and it's intended to remove the need to create
custom escaping code and offer an even more secure platform to utilise
in building your applications with Zend Framework 2.0.

Your feedback is, as always, extremely welcome whether by comment or a
reply email.

Best regards,
Paddy

--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team


Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Pádraic Brady
In reply to this post by Stewart Lord
On Wed, Feb 29, 2012 at 10:03 PM, Stewart Lord <[hidden email]> wrote:
> Hi Paddy,
>
> This is great. I wonder if we could get this into ZF1? We have implemented
> escapeJs() and escapeAttr() helpers (which you are welcome to see) and it
> would be great if there were official versions that were potentially faster
> or more secure. In any case I could backport your ZF2 versions for my own
> use if they are better.

It's probably unlikely to be backported but if the class itself is
kept decoupled it should be reusable regardless ;).

> Our helpers rely on iconv instead of mbstring. It's been a while since I've
> worked on it, so I can't recall why we made that decision. I will say that
> our version does character-by-character iteration and profiling indicates
> performance is not great.

Performance isn't great - I have helpers here myself. I'm
experimenting with multibyte character replacements from a
pre-generated blacklist (the character ranges for escaping are well
defined) to see if that makes any difference. It's still very fugly
but even a modest performance increase would be worthwhile. The
blacklist would be compared regularly to a generated equivelant in the
unit tests - i.e. we'll keep an eye on it since we all know blacklists
like to fall out of date over time ;).

> Regarding your plan to put these under Zend\View\Escaper, did you consider
> making these filters?

For now, the RFC assumes it will be a discrete sub-component in the
Zend\View namespace. From there it would be composable by Zend\View
classes or filter classes as needed. I'd prefer it to be a single
concrete entity so it's less bound to ZF2's concept of Views, filters,
and helpers etc., and can be reused or integrated, with a bit of luck,
by those building in support for the likes of Twig and Smarty - not to
mention where else in the framework escaping would come in handy.

--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

xoops
+1 for building an independent component for escaper instead of inside view component.
Furthermore, perhaps a more generic text processing component is desired including escaper.

On Thu, Mar 1, 2012 at 8:00 AM, Pádraic Brady <[hidden email]> wrote:
On Wed, Feb 29, 2012 at 10:03 PM, Stewart Lord <[hidden email]> wrote:
> Hi Paddy,
>
> This is great. I wonder if we could get this into ZF1? We have implemented
> escapeJs() and escapeAttr() helpers (which you are welcome to see) and it
> would be great if there were official versions that were potentially faster
> or more secure. In any case I could backport your ZF2 versions for my own
> use if they are better.

It's probably unlikely to be backported but if the class itself is
kept decoupled it should be reusable regardless ;).

> Our helpers rely on iconv instead of mbstring. It's been a while since I've
> worked on it, so I can't recall why we made that decision. I will say that
> our version does character-by-character iteration and profiling indicates
> performance is not great.

Performance isn't great - I have helpers here myself. I'm
experimenting with multibyte character replacements from a
pre-generated blacklist (the character ranges for escaping are well
defined) to see if that makes any difference. It's still very fugly
but even a modest performance increase would be worthwhile. The
blacklist would be compared regularly to a generated equivelant in the
unit tests - i.e. we'll keep an eye on it since we all know blacklists
like to fall out of date over time ;).

> Regarding your plan to put these under Zend\View\Escaper, did you consider
> making these filters?

For now, the RFC assumes it will be a discrete sub-component in the
Zend\View namespace. From there it would be composable by Zend\View
classes or filter classes as needed. I'd prefer it to be a single
concrete entity so it's less bound to ZF2's concept of Views, filters,
and helpers etc., and can be reused or integrated, with a bit of luck,
by those building in support for the likes of Twig and Smarty - not to
mention where else in the framework escaping would come in handy.

--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team



--

Taiwen Jiang (aka D.J.)

Build Xoops Engine
web and mobile application platform

CTO for EEFOCUS.com
Leading social platform for electronics professionals


Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Mike Willbanks
In reply to this post by Pádraic Brady
Padraic,

> I've added an Escaper Class RFC to the list of things you are going to
> read this week. I know you'll all read it and provide valuable
> feedback because you like building secure web applications. You
> probably also want to continue living which will quickly cease if you
> don't read this. Final warning, I have a whole team of ninja assassins
> ready to be dispatched ;).

Always threatening aren't you; funny funny man.

>
> http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class
>
> The RFC proposes a new class designed to offer secure escaping for
> Zend\View beyond basic htmlspecialchars() usage. Its stated goal is to
> implement contextual escaping based on peer-reviewed rules that carry
> considerable weight as a recommended practice. In other words, I'm
> relying on the published OWASP ESAPI codecs for guidance (you can find
> the Open Web Application Security Project (OWASP) at
> https://www.owasp.org but I'm sure you already have it bookmarked!).
> As security problems, particularly Cross Site Scripting, have
> continued to plague PHP applications, this approach goes a lot further
> than you might suspect and it's intended to remove the need to create
> custom escaping code and offer an even more secure platform to utilise
> in building your applications with Zend Framework 2.0.

I would be particularly interested in some more details on how the
actual parsing of this works in terms of the ESAPI; there is not a ton
of documentation but more or less a java library that gives a better
explanation.  It would be good to also link the documentation on some
of the design for this.  I do highly enjoy the fact that it is
contextual (something PHP significantly lacks).

I do think that the API should allow for specifically stating which
type of attributes to allow through the filtering / transformation
process; much like html sanitizer allows us - especially since this is
a common use case.  They will likely shoot themselves in the foot at
times because some people won't recognize the fact that they should be
very very careful about the attributes that they are allowing.

Performance Concerns:
I believe that there is ways around this; such as ensuring that we
teach others which type of output they should be rendering (aka not a
whole string they put together but the dangerous output they are
integrating from user input).  Additionally there might be an
opportunity to add a caching layer on top through the use of events;
more specifically there are likely common types of output that could
benefit from a temporary content cache.  For instance; if I am using a
persons name on each page they visit; it might make sense for that to
be cached (however, hopefully I already took care of the filtering and
validation prior to allowing it in my data store).



Regards,

Mike
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

weierophinney
Administrator
In reply to this post by Pádraic Brady
-- Pádraic Brady <[hidden email]> wrote
(on Thursday, 01 March 2012, 12:00 AM +0000):

> On Wed, Feb 29, 2012 at 10:03 PM, Stewart Lord <[hidden email]> wrote:
> > Regarding your plan to put these under Zend\View\Escaper, did you consider
> > making these filters?
>
> For now, the RFC assumes it will be a discrete sub-component in the
> Zend\View namespace. From there it would be composable by Zend\View
> classes or filter classes as needed. I'd prefer it to be a single
> concrete entity so it's less bound to ZF2's concept of Views, filters,
> and helpers etc., and can be reused or integrated, with a bit of luck,
> by those building in support for the likes of Twig and Smarty - not to
> mention where else in the framework escaping would come in handy.

Perhaps a top-level namespace such as Zend\Security or Zend\Escape might
be better in that regard, then -- makes it simpler to package. :)

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Andreas Möller
In reply to this post by Pádraic Brady

<snip>

OT: I remember times in the Usenet where one would get *plonk*ed for top-posting. 

http://www.dickgaughan.co.uk/usenet/guide/faq08-topp.html


Andreas
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

xoops
In reply to this post by weierophinney
Vote for Zend\Security which I had built one for ZF1.

On Thu, Mar 1, 2012 at 12:38 PM, Matthew Weier O'Phinney <[hidden email]> wrote:
-- Pádraic Brady <[hidden email]> wrote
(on Thursday, 01 March 2012, 12:00 AM +0000):
> On Wed, Feb 29, 2012 at 10:03 PM, Stewart Lord <[hidden email]> wrote:
> > Regarding your plan to put these under Zend\View\Escaper, did you consider
> > making these filters?
>
> For now, the RFC assumes it will be a discrete sub-component in the
> Zend\View namespace. From there it would be composable by Zend\View
> classes or filter classes as needed. I'd prefer it to be a single
> concrete entity so it's less bound to ZF2's concept of Views, filters,
> and helpers etc., and can be reused or integrated, with a bit of luck,
> by those building in support for the likes of Twig and Smarty - not to
> mention where else in the framework escaping would come in handy.

Perhaps a top-level namespace such as Zend\Security or Zend\Escape might
be better in that regard, then -- makes it simpler to package. :)

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc



--

Taiwen Jiang (aka D.J.)

Build Xoops Engine
web and mobile application platform

CTO for EEFOCUS.com
Leading social platform for electronics professionals


Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Stewart Lord
In reply to this post by Pádraic Brady
On 2/29/12 4:00 PM, Pádraic Brady wrote:
> I'm experimenting with multibyte character replacements from a
> pre-generated blacklist (the character ranges for escaping are well
> defined) to see if that makes any difference.

Are you assuming UTF8 then? If we assume (or normalize to) UTF8 we might
be able to cut some corners (such as using preg_replace with 'u'?).
Otherwise I guess we need to use mb_substr, mb_split or iconv_substr to
identify character boundaries correctly which is expensive.

Stew
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Stewart Lord
In reply to this post by Mike Willbanks
On 2/29/12 5:37 PM, Mike Willbanks wrote:
> I would be particularly interested in some more details on how the
> actual parsing of this works in terms of the ESAPI;
> (snip)
> I do think that the API should allow for specifically stating which
> type of attributes to allow through the filtering / transformation
> process;


Hi Mike,

The escaping functions don't really do any parsing or deal with
attributes. They just filter strings to ensure unsafe characters are
escaped.

It is up to the developer to use the function when printing an attribute
value. For example:

  <input type="hidden" value="<?= $this->escapeAttr($data) ?>">

Regards,
Stew
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Artur Bodera
In reply to this post by Pádraic Brady
On Wed, Feb 29, 2012 at 10:48 PM, Pádraic Brady <[hidden email]> wrote:

http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class




A few points from me:

Performance degradation due to missing extensions is a common and understandable scenario in PHP. I do not feel that we should try to improve performance at cost of security at ANY point -- given we're talking about security-related component.

That said, the component should "gracefuly degradate" (performance wise) when there are extensions missing. We can have that using factories and subclasses for wrapping around missing extensions. For example (simplification):

    new Escaper()
        Engine::getInstance()
            if ( extension_exists('mbstring') ){  new Engine\Mbstring ...
            elseif ( function_exists('iconv') ) {  new Engine\Iconv ...
            elseif (e xtension_exists('recode') ){  new Engine\Recode ...
            else {  new Engine\Fallback...


That means:

 1) we get _maximum_ security in given context

 2) Depending on loaded extensions, performance will vary.

 3) zf performance measurements will be harder, but in 
     2012 any decent php developer should know that 
     php installations and their performance will vary and 
     have strong impact on any analytic benchmark.

 4) The decision to sacrifice security to gain some performance
     is FOR THE DEVELOPER to make, not us. If one decides, that
     Escaper component (even with mbstring) is too slow, he/she
     can switch back to htmlentities, addslashes etc. 

 4) If one uses Escaper, one can assume that at some cost of 
     performance one's getting best available and accurate 
     sanitization that secures against (some) exploits.


I'm all in to entertain OWASP recommendations with all steps they provide. That should be archived regardless of performance loss. It will result in an anti-XSS component, or at least one that brings us one step closer to that goal. That said, the component should be available as a Filter, or otherwise to make it usable in non-View scenario (i.e. when manually composing email messages or sending Jabber messages or whatever).

There's also a question of reasonable coupling. For example, we've recently mentioned addslashes() being dumb and not being able to determine if it's being used in HTML5 or HTML4 scenario, which differ by the usage of [ ' ]  char for attributes (i.e. HTML5 allows for <input name='quagmire' ... while HTML4/XHTML does not). My recommendation for that is to create a configurable Escaper (i.e. as Filter) to be used standalone + a series of view helpers that are coupled with View layer AND other helpers. That means, they can query Doctype view helper and determine doctype, or work with other parts of MVC to better serve their purpose.

Thanks for reading.



A.

btw: excellent job on the RFC. You're a master of words. Great read :) 

-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




 
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Pádraic Brady
On Thu, Mar 1, 2012 at 11:32 AM, Artur Bodera <[hidden email]> wrote:
> A few points from me:

Ninja assassin recalled. He's very disappointed ;)

> Performance degradation due to missing extensions is a common and
> understandable scenario in PHP. I do not feel that we should try to improve
> performance at cost of security at ANY point -- given we're talking about
> security-related component.

I have the same opinion but as the RFC notes, I personally believe
that if it's too slow users will simply not use it and then will run
away to stare at intra-framework benchmarks to justify their decision.
In PHP we have wholly unreasonable expectations of how security
defenses should perform.

> That said, the component should "gracefuly degradate" (performance wise)
> when there are extensions missing. We can have that using factories and
> subclasses for wrapping around missing extensions. For example
> (simplification):
>
>     new Escaper()
>         Engine::getInstance()
>             if ( extension_exists('mbstring') ){  new Engine\Mbstring ...
>             elseif ( function_exists('iconv') ) {  new Engine\Iconv ...
>             elseif (e xtension_exists('recode') ){  new Engine\Recode ...
>             else {  new Engine\Fallback...

I appreciate the point ;). The problem is taking the adapter route is
one level of complexity from where I'd like to be. The Javascript, CSS
and Url escapers will perform character-based parsing anyway (there's
no PHP function support for those) and the HTML escapers would use
htmlspecialchars(). From there, we have two routes. With mbstring
(preferably) enabled, we can implement safer escaping and also
possibly speed them up.

As that suggests it's really difficult to make non-mbstring/iconv
supporting servers fully secure since we must have multibyte character
support to handle encoding issues and cleanup. There can be no
reasonable fallback to this without enforcing even worse performance
on users - and then they just won't use the escapers at all. In
effect, this is the same as telling users that having mbstring
available is good for security which is the message I'm happy to send.

> That means:
>
>  1) we get _maximum_ security in given context
>
>  2) Depending on loaded extensions, performance will vary.

Again, maximum security basically requires one of the extensions. If
none are available, you have issues anyway since it means you're
already not doing character encoding cleanup.

>  3) zf performance measurements will be harder, but in
>      2012 any decent php developer should know that
>      php installations and their performance will vary and
>      have strong impact on any analytic benchmark.

I give it a few weeks before someone runs benchmarks on a laptop ;).
Question is what would they compare the Escaper to?

>  4) The decision to sacrifice security to gain some performance
>      is FOR THE DEVELOPER to make, not us. If one decides, that
>      Escaper component (even with mbstring) is too slow, he/she
>      can switch back to htmlentities, addslashes etc.

In a sense they still have that choice since they can not use the
stricter modes. But as I said earlier, the only escaper having a
significant performance loss by being strict would be the HTML
escapers. The rest would be secure by default since there are no
performance optimisations possible without multibyte character support
(and that's based on a gut feeling while I run some tests here).

> I'm all in to entertain OWASP recommendations with all steps they provide.
> That should be archived regardless of performance loss. It will result in an
> anti-XSS component, or at least one that brings us one step closer to that
> goal. That said, the component should be available as a Filter, or otherwise
> to make it usable in non-View scenario (i.e. when manually composing email
> messages or sending Jabber messages or whatever).

Basically, I would prefer this be one single component which can be
composed into anything you might like ;). The reason for keeping
aggregating to one class (or namespace) is that, as a security
library, it makes it easier to port to other non-ZF applications
without dragging in too many unrelated dependencies.

> There's also a question of reasonable coupling. For example, we've recently
> mentioned addslashes() being dumb and not being able to determine if it's
> being used in HTML5 or HTML4 scenario, which differ by the usage of [ ' ]
>  char for attributes (i.e. HTML5 allows for <input name='quagmire' ... while
> HTML4/XHTML does not). My recommendation for that is to create a
> configurable Escaper (i.e. as Filter) to be used standalone + a series of
> view helpers that are coupled with View layer AND other helpers. That means,
> they can query Doctype view helper and determine doctype, or work with other
> parts of MVC to better serve their purpose.

This is moving into the territory of becoming context-aware which has
one important outcome - the framework gains responsibility better left
with users. The great flaw (and I could use stronger words) of
context-detecting escapers is that they don't work. They can figure
out the right kind of escaping for one context level but nesting
contexts quickly confuse them. We should be careful not to stray in
that direction unless we're willing to go the whole way.

There's no harm in raising ideas where this would add real
improvements without creating a false sense of security but they're
not addressed in this RFC ;).

> Thanks for reading.

Thanks for writing!

> A.
>
> btw: excellent job on the RFC. You're a master of words. Great read :)

Do I win something? I could do with more money!

Paddy
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Pádraic Brady
In reply to this post by Stewart Lord
On Thu, Mar 1, 2012 at 6:20 AM, Stewart Lord <[hidden email]> wrote:

> On 2/29/12 4:00 PM, Pádraic Brady wrote:
>>
>> I'm experimenting with multibyte character replacements from a
>> pre-generated blacklist (the character ranges for escaping are well
>> defined) to see if that makes any difference.
>
> Are you assuming UTF8 then? If we assume (or normalize to) UTF8 we might be
> able to cut some corners (such as using preg_replace with 'u'?). Otherwise I
> guess we need to use mb_substr, mb_split or iconv_substr to identify
> character boundaries correctly which is expensive.

The plan is to normalise to UTF-8 wherever possible - it just requires
mbstring or iconv to support that so the new escapers are necessarily
limited to ASCII compatible encodings otherwise (a fair divergence
from htmlspecialchars() support).

Whether it uses preg_replace or not, I'm not sure. There are
advantages and disadvantages to each approach and it's probably better
to use something more rudimentary until we at least have a good unit
test suite to support refactoring and testing of the possibly better
performing methods such as preg_replace() (assuming we can get over
it's own treatment of possibly naughty unicode).


--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Marc Bennewitz (private)
In reply to this post by Pádraic Brady

On 01.03.2012 16:39, Pádraic Brady [via Zend Framework Community] wrote:
On Thu, Mar 1, 2012 at 11:32 AM, Artur Bodera <[hidden email]> wrote:
> A few points from me:

Ninja assassin recalled. He's very disappointed ;)

> Performance degradation due to missing extensions is a common and
> understandable scenario in PHP. I do not feel that we should try to improve
> performance at cost of security at ANY point -- given we're talking about
> security-related component.

I have the same opinion but as the RFC notes, I personally believe
that if it's too slow users will simply not use it and then will run
away to stare at intra-framework benchmarks to justify their decision.
In PHP we have wholly unreasonable expectations of how security
defenses should perform.
If it is too slow it should make sense to write an extension for it which could be used
also by users not ( or don't like to ) use ZF.


> That said, the component should "gracefuly degradate" (performance wise)
> when there are extensions missing. We can have that using factories and
> subclasses for wrapping around missing extensions. For example
> (simplification):
>
>     new Escaper()
>         Engine::getInstance()
>             if ( extension_exists('mbstring') ){  new Engine\Mbstring ...
>             elseif ( function_exists('iconv') ) {  new Engine\Iconv ...
>             elseif (e xtension_exists('recode') ){  new Engine\Recode ...
>             else {  new Engine\Fallback...

I appreciate the point ;). The problem is taking the adapter route is
one level of complexity from where I'd like to be. The Javascript, CSS
and Url escapers will perform character-based parsing anyway (there's
no PHP function support for those) and the HTML escapers would use
htmlspecialchars(). From there, we have two routes. With mbstring
(preferably) enabled, we can implement safer escaping and also
possibly speed them up.

As that suggests it's really difficult to make non-mbstring/iconv
supporting servers fully secure since we must have multibyte character
support to handle encoding issues and cleanup. There can be no
reasonable fallback to this without enforcing even worse performance
on users - and then they just won't use the escapers at all. In
effect, this is the same as telling users that having mbstring
available is good for security which is the message I'm happy to send.

> That means:
>
>  1) we get _maximum_ security in given context
>
>  2) Depending on loaded extensions, performance will vary.

Again, maximum security basically requires one of the extensions. If
none are available, you have issues anyway since it means you're
already not doing character encoding cleanup.

>  3) zf performance measurements will be harder, but in
>      2012 any decent php developer should know that
>      php installations and their performance will vary and
>      have strong impact on any analytic benchmark.

I give it a few weeks before someone runs benchmarks on a laptop ;).
Question is what would they compare the Escaper to?

>  4) The decision to sacrifice security to gain some performance
>      is FOR THE DEVELOPER to make, not us. If one decides, that
>      Escaper component (even with mbstring) is too slow, he/she
>      can switch back to htmlentities, addslashes etc.

In a sense they still have that choice since they can not use the
stricter modes. But as I said earlier, the only escaper having a
significant performance loss by being strict would be the HTML
escapers. The rest would be secure by default since there are no
performance optimisations possible without multibyte character support
(and that's based on a gut feeling while I run some tests here).

> I'm all in to entertain OWASP recommendations with all steps they provide.
> That should be archived regardless of performance loss. It will result in an
> anti-XSS component, or at least one that brings us one step closer to that
> goal. That said, the component should be available as a Filter, or otherwise
> to make it usable in non-View scenario (i.e. when manually composing email
> messages or sending Jabber messages or whatever).

Basically, I would prefer this be one single component which can be
composed into anything you might like ;). The reason for keeping
aggregating to one class (or namespace) is that, as a security
library, it makes it easier to port to other non-ZF applications
without dragging in too many unrelated dependencies.

> There's also a question of reasonable coupling. For example, we've recently
> mentioned addslashes() being dumb and not being able to determine if it's
> being used in HTML5 or HTML4 scenario, which differ by the usage of [ ' ]
>  char for attributes (i.e. HTML5 allows for <input name='quagmire' ... while
> HTML4/XHTML does not). My recommendation for that is to create a
> configurable Escaper (i.e. as Filter) to be used standalone + a series of
> view helpers that are coupled with View layer AND other helpers. That means,
> they can query Doctype view helper and determine doctype, or work with other
> parts of MVC to better serve their purpose.

This is moving into the territory of becoming context-aware which has
one important outcome - the framework gains responsibility better left
with users. The great flaw (and I could use stronger words) of
context-detecting escapers is that they don't work. They can figure
out the right kind of escaping for one context level but nesting
contexts quickly confuse them. We should be careful not to stray in
that direction unless we're willing to go the whole way.

There's no harm in raising ideas where this would add real
improvements without creating a false sense of security but they're
not addressed in this RFC ;).

> Thanks for reading.

Thanks for writing!

> A.
>
> btw: excellent job on the RFC. You're a master of words. Great read :)

Do I win something? I could do with more money!

Paddy



To start a new topic under ZF Contributor, email [hidden email]
To unsubscribe from ZF Contributor, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Anton Stöckl
In reply to this post by Pádraic Brady
Hi Pádraic!

I've really _never_ been into Glam Metal bands, so "OWASP" seems a bit
suspicious ... ;-)

Ok, jokes aside. I got a detail question:

escapeUri()

I'm a german so I regularly have to deal with german content which
obviously contains many non ascii chars. I've had some good fun building
a filter that transliterates that to ascii so it can be used as a slug
in a url. For german chars this works well with a combination of mb_
functions and iconv with 'ASCII//TRANSLIT//IGNORE' and then doing some
preg_replace for whitespaces and whatever is left over.

We have UGC content and everything is UTF-8. Recently some guy created
content that was completely Greek characters (could also have been
Cyrillic or whatnot).

I ended up with a lookup table for such characters as a first filter,
which is obviously more a hack than a solution.

There are other places where you need to translit any UTF-8 to ascii,
tags for the TwoLayer cache backends for example.

So I wonder if:
- the new escaper class adresses that topic
- which "strategy" this would fit in, it's not just for URLs

Best regards, Anton



Am 29.02.2012 22:48, schrieb Pádraic Brady:

> Hi all,
>
> I've added an Escaper Class RFC to the list of things you are going to
> read this week. I know you'll all read it and provide valuable
> feedback because you like building secure web applications. You
> probably also want to continue living which will quickly cease if you
> don't read this. Final warning, I have a whole team of ninja assassins
> ready to be dispatched ;).
>
> http://framework.zend.com/wiki/display/ZFDEV2/RFC+-+Escaper+Class
>
> The RFC proposes a new class designed to offer secure escaping for
> Zend\View beyond basic htmlspecialchars() usage. Its stated goal is to
> implement contextual escaping based on peer-reviewed rules that carry
> considerable weight as a recommended practice. In other words, I'm
> relying on the published OWASP ESAPI codecs for guidance (you can find
> the Open Web Application Security Project (OWASP) at
> https://www.owasp.org but I'm sure you already have it bookmarked!).
> As security problems, particularly Cross Site Scripting, have
> continued to plague PHP applications, this approach goes a lot further
> than you might suspect and it's intended to remove the need to create
> custom escaping code and offer an even more secure platform to utilise
> in building your applications with Zend Framework 2.0.
>
> Your feedback is, as always, extremely welcome whether by comment or a
> reply email.
>
> Best regards,
> Paddy
>

Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Artur Bodera
On Fri, Mar 2, 2012 at 9:49 AM, Anton Stöckl <[hidden email]> wrote:
So I wonder if:
- the new escaper class adresses that topic
- which "strategy" this would fit in, it's not just for URLs


FYI:



Cheers Ben! :)


A. 
Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Anton Stöckl
Cheers Artur, I'll give it a try!
Did not stumple over this project in my researches (sadly).
So, you have used this library and are sure it's safe?

-Anton

Am 02.03.2012 10:27, schrieb Artur Bodera:

> On Fri, Mar 2, 2012 at 9:49 AM, Anton Stöckl <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     So I wonder if:
>     - the new escaper class adresses that topic
>     - which "strategy" this would fit in, it's not just for URLs
>
>
> FYI:
>
> https://github.com/DASPRiD/Bacon/tree/master/src/Bacon/Text/UniDecode
>
>
> Cheers Ben! :)
>
>
> A.

Reply | Threaded
Open this post in threaded view
|

Re: RFC - Escaper Class (Extended Escaping for Zend\View)

Artur Bodera
On Fri, Mar 2, 2012 at 10:49 AM, Anton Stöckl <[hidden email]> wrote:
Cheers Artur, I'll give it a try!
Did not stumple over this project in my researches (sadly).
So, you have used this library and are sure it's safe?

No, haven't tried it, but it's a proof of concept - it's possible to do without iconv//TRANSLIT.

I don't think it's 100% safe and I'd recommend against using it yet.

We could use parts of it though to build Escaper component (Paddy, are you getting that? :-) )



-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




 
12