Quantcast

View Layer: lingering questions

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

View Layer: lingering questions

weierophinney
Administrator
Hey, all!

Last Friday, I submitted the pull request for the view layer:

    https://github.com/zendframework/zf2/pull/801

I've had some back and forth with Rob Allen as he reviews it for
inclusion, but we're almost ready at this point.

One lingering question remains, one that isn't directly related to the
scope of the RFC, but which I'm interested in answering anyways:

    Should we provide auto-escaping by default within the PhpRenderer?

The reason I ask is this:

 * In ZF1, you knew that if you didn't escape() a variable, you had
   better have a good reason.
 * In ZF2, _SCALAR_ variables will be escaped by default, but variables
   within arrays or objects are not. So, now you need to know that you
   need to escape array/object members, but not others.

As others have pointed out, this second point actually can be a bit
harder to explain to users, and may lead to WTF?!?!?! moments when they
assume "echo $this->someObject->property" will be escaped, and is not.

The "solution" to escaping is to:

 * introduce context-specific escaping, which would require parsing and
   compiling templates, OR
 * running the final output through a sanitiser of some sort

Both of which introduce complexity and degrade performance (though
performance can be mitigated by good caching, obviously).

In the meantime, however, my question stands:

    Should we provide auto-escaping by default within the PhpRenderer?

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Pádraic Brady
Hi Matthew,

Option 1 might work but I'd advise against Option 2 - sanitising HTML
ain't pretty or easy and, done properly, it's very very slow.

Paddy

On Tue, Feb 21, 2012 at 6:51 PM, Matthew Weier O'Phinney
<[hidden email]> wrote:

> Hey, all!
>
> Last Friday, I submitted the pull request for the view layer:
>
>    https://github.com/zendframework/zf2/pull/801
>
> I've had some back and forth with Rob Allen as he reviews it for
> inclusion, but we're almost ready at this point.
>
> One lingering question remains, one that isn't directly related to the
> scope of the RFC, but which I'm interested in answering anyways:
>
>    Should we provide auto-escaping by default within the PhpRenderer?
>
> The reason I ask is this:
>
>  * In ZF1, you knew that if you didn't escape() a variable, you had
>   better have a good reason.
>  * In ZF2, _SCALAR_ variables will be escaped by default, but variables
>   within arrays or objects are not. So, now you need to know that you
>   need to escape array/object members, but not others.
>
> As others have pointed out, this second point actually can be a bit
> harder to explain to users, and may lead to WTF?!?!?! moments when they
> assume "echo $this->someObject->property" will be escaped, and is not.
>
> The "solution" to escaping is to:
>
>  * introduce context-specific escaping, which would require parsing and
>   compiling templates, OR
>  * running the final output through a sanitiser of some sort
>
> Both of which introduce complexity and degrade performance (though
> performance can be mitigated by good caching, obviously).
>
> In the meantime, however, my question stands:
>
>    Should we provide auto-escaping by default within the PhpRenderer?
>
> --
> Matthew Weier O'Phinney
> Project Lead            | [hidden email]
> Zend Framework          | http://framework.zend.com/
> PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc



--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Marco Pivetta
This post has NOT been accepted by the mailing list yet.
I don't want that to be part of the defaults.
It is counter-intuitive and doesn't fit all the problems of escaping in various contexts :(
That is up to the developer.
There's lots of place in Zend\Filter and in view helpers for that :)

Marco Pivetta

http://twitter.com/Ocramius     

http://marco-pivetta.com    



On 21 February 2012 20:01, Pádraic Brady [via Zend Framework Community] <[hidden email]> wrote:
Hi Matthew,

Option 1 might work but I'd advise against Option 2 - sanitising HTML
ain't pretty or easy and, done properly, it's very very slow.

Paddy

On Tue, Feb 21, 2012 at 6:51 PM, Matthew Weier O'Phinney
<[hidden email]> wrote:

> Hey, all!
>
> Last Friday, I submitted the pull request for the view layer:
>
>    https://github.com/zendframework/zf2/pull/801
>
> I've had some back and forth with Rob Allen as he reviews it for
> inclusion, but we're almost ready at this point.
>
> One lingering question remains, one that isn't directly related to the
> scope of the RFC, but which I'm interested in answering anyways:
>
>    Should we provide auto-escaping by default within the PhpRenderer?
>
> The reason I ask is this:
>
>  * In ZF1, you knew that if you didn't escape() a variable, you had
>   better have a good reason.
>  * In ZF2, _SCALAR_ variables will be escaped by default, but variables
>   within arrays or objects are not. So, now you need to know that you
>   need to escape array/object members, but not others.
>
> As others have pointed out, this second point actually can be a bit
> harder to explain to users, and may lead to WTF?!?!?! moments when they
> assume "echo $this->someObject->property" will be escaped, and is not.
>
> The "solution" to escaping is to:
>
>  * introduce context-specific escaping, which would require parsing and
>   compiling templates, OR
>  * running the final output through a sanitiser of some sort
>
> Both of which introduce complexity and degrade performance (though
> performance can be mitigated by good caching, obviously).
>
> In the meantime, however, my question stands:
>
>    Should we provide auto-escaping by default within the PhpRenderer?
>
> --
> Matthew Weier O'Phinney
> Project Lead            | [hidden email]



--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team



If you reply to this email, your message will be added to the discussion below:
http://zend-framework-community.634137.n4.nabble.com/View-Layer-lingering-questions-tp4407918p4407942.html
To start a new topic under ZF Contributor, email [hidden email]
To unsubscribe from ZF Contributor, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Marco Pivetta (Ocramius)
In reply to this post by Pádraic Brady
I don't want that to be part of the defaults.
It is counter-intuitive and doesn't handle all the problems of escaping in various contexts :(
That is up to the developer in my opinion.
There's lots of place in Zend\Filter and in view helpers for that :)

2012/2/21 Pádraic Brady <[hidden email]>
Hi Matthew,

Option 1 might work but I'd advise against Option 2 - sanitising HTML
ain't pretty or easy and, done properly, it's very very slow.

Paddy

On Tue, Feb 21, 2012 at 6:51 PM, Matthew Weier O'Phinney
<[hidden email]> wrote:
> Hey, all!
>
> Last Friday, I submitted the pull request for the view layer:
>
>    https://github.com/zendframework/zf2/pull/801
>
> I've had some back and forth with Rob Allen as he reviews it for
> inclusion, but we're almost ready at this point.
>
> One lingering question remains, one that isn't directly related to the
> scope of the RFC, but which I'm interested in answering anyways:
>
>    Should we provide auto-escaping by default within the PhpRenderer?
>
> The reason I ask is this:
>
>  * In ZF1, you knew that if you didn't escape() a variable, you had
>   better have a good reason.
>  * In ZF2, _SCALAR_ variables will be escaped by default, but variables
>   within arrays or objects are not. So, now you need to know that you
>   need to escape array/object members, but not others.
>
> As others have pointed out, this second point actually can be a bit
> harder to explain to users, and may lead to WTF?!?!?! moments when they
> assume "echo $this->someObject->property" will be escaped, and is not.
>
> The "solution" to escaping is to:
>
>  * introduce context-specific escaping, which would require parsing and
>   compiling templates, OR
>  * running the final output through a sanitiser of some sort
>
> Both of which introduce complexity and degrade performance (though
> performance can be mitigated by good caching, obviously).
>
> In the meantime, however, my question stands:
>
>    Should we provide auto-escaping by default within the PhpRenderer?
>
> --
> Matthew Weier O'Phinney
> Project Lead            | [hidden email]
> Zend Framework          | http://framework.zend.com/
> PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc



--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Kim Blomqvist
In reply to this post by weierophinney
Hello Matthew,

Don't know if this can work, but how about introducing Escaper class for
someObjects assigned to the view? The Escaper would work as a wrapper or
a proxy class for the someObject and would escape what ever is fetched
from the object. This could be completely invisible for the user as the
"wrapping" can happen when someObject is assigned to the view.

Regards,
Kim

21.2.2012 20:51, Matthew Weier O'Phinney kirjoitti:

> Hey, all!
>
> Last Friday, I submitted the pull request for the view layer:
>
>     https://github.com/zendframework/zf2/pull/801
>
> I've had some back and forth with Rob Allen as he reviews it for
> inclusion, but we're almost ready at this point.
>
> One lingering question remains, one that isn't directly related to the
> scope of the RFC, but which I'm interested in answering anyways:
>
>     Should we provide auto-escaping by default within the PhpRenderer?
>
> The reason I ask is this:
>
>  * In ZF1, you knew that if you didn't escape() a variable, you had
>    better have a good reason.
>  * In ZF2, _SCALAR_ variables will be escaped by default, but variables
>    within arrays or objects are not. So, now you need to know that you
>    need to escape array/object members, but not others.
>
> As others have pointed out, this second point actually can be a bit
> harder to explain to users, and may lead to WTF?!?!?! moments when they
> assume "echo $this->someObject->property" will be escaped, and is not.
>
> The "solution" to escaping is to:
>
>  * introduce context-specific escaping, which would require parsing and
>    compiling templates, OR
>  * running the final output through a sanitiser of some sort
>
> Both of which introduce complexity and degrade performance (though
> performance can be mitigated by good caching, obviously).
>
> In the meantime, however, my question stands:
>
>     Should we provide auto-escaping by default within the PhpRenderer?
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Tomáš Fejfar
In reply to this post by weierophinney
The auto-escape implementation in PhpRenderer (as of beta2) is bad and misleading and should NOT be there. Same goes to auto-extracting (as nice as it in theory sounds) unless there is some middle layer that allows completly different syntax then normal PHP variable syntax that cannot be confused for local variable... 

Content-aware escaping would be a lovely feature and proper caching should be an easy opt-in (opt-out even better but I understand it's politically difficult to do in library code). 


On Tue, Feb 21, 2012 at 7:51 PM, Matthew Weier O'Phinney <[hidden email]> wrote:
Hey, all!

Last Friday, I submitted the pull request for the view layer:

   https://github.com/zendframework/zf2/pull/801

I've had some back and forth with Rob Allen as he reviews it for
inclusion, but we're almost ready at this point.

One lingering question remains, one that isn't directly related to the
scope of the RFC, but which I'm interested in answering anyways:

   Should we provide auto-escaping by default within the PhpRenderer?

The reason I ask is this:

 * In ZF1, you knew that if you didn't escape() a variable, you had
  better have a good reason.
 * In ZF2, _SCALAR_ variables will be escaped by default, but variables
  within arrays or objects are not. So, now you need to know that you
  need to escape array/object members, but not others.

As others have pointed out, this second point actually can be a bit
harder to explain to users, and may lead to WTF?!?!?! moments when they
assume "echo $this->someObject->property" will be escaped, and is not.

The "solution" to escaping is to:

 * introduce context-specific escaping, which would require parsing and
  compiling templates, OR
 * running the final output through a sanitiser of some sort

Both of which introduce complexity and degrade performance (though
performance can be mitigated by good caching, obviously).

In the meantime, however, my question stands:

   Should we provide auto-escaping by default within the PhpRenderer?

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

weierophinney
Administrator
In reply to this post by Pádraic Brady
-- Pádraic Brady <[hidden email]> wrote
(on Tuesday, 21 February 2012, 07:01 PM +0000):
> Option 1 might work

It's the "might" that bothers me; even with a good parser/compiler,
you're likely to have holes/edge cases where it fails, and that can lead
to very serious security issues.

> but I'd advise against Option 2 - sanitising HTML
> ain't pretty or easy and, done properly, it's very very slow.

Oh, of that, I'm well aware. :)

So, is this a vote for "remove auto-escaping" in the current
PhpRenderer, or a vote for a full-on templating engine, ala Twig,
Smarty, etc.?

> On Tue, Feb 21, 2012 at 6:51 PM, Matthew Weier O'Phinney
> <[hidden email]> wrote:
> > Hey, all!
> >
> > Last Friday, I submitted the pull request for the view layer:
> >
> >    https://github.com/zendframework/zf2/pull/801
> >
> > I've had some back and forth with Rob Allen as he reviews it for
> > inclusion, but we're almost ready at this point.
> >
> > One lingering question remains, one that isn't directly related to the
> > scope of the RFC, but which I'm interested in answering anyways:
> >
> >    Should we provide auto-escaping by default within the PhpRenderer?
> >
> > The reason I ask is this:
> >
> >  * In ZF1, you knew that if you didn't escape() a variable, you had
> >   better have a good reason.
> >  * In ZF2, _SCALAR_ variables will be escaped by default, but variables
> >   within arrays or objects are not. So, now you need to know that you
> >   need to escape array/object members, but not others.
> >
> > As others have pointed out, this second point actually can be a bit
> > harder to explain to users, and may lead to WTF?!?!?! moments when they
> > assume "echo $this->someObject->property" will be escaped, and is not.
> >
> > The "solution" to escaping is to:
> >
> >  * introduce context-specific escaping, which would require parsing and
> >   compiling templates, OR
> >  * running the final output through a sanitiser of some sort
> >
> > Both of which introduce complexity and degrade performance (though
> > performance can be mitigated by good caching, obviously).
> >
> > In the meantime, however, my question stands:
> >
> >    Should we provide auto-escaping by default within the PhpRenderer?
> >
> > --
> > Matthew Weier O'Phinney
> > Project Lead            | [hidden email]
> > Zend Framework          | http://framework.zend.com/
> > PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
>
>
>
> --
> Pádraic Brady
>
> http://blog.astrumfutura.com
> http://www.survivethedeepend.com
> Zend Framework Community Review Team
>

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Artur Bodera
In reply to this post by weierophinney
On Tue, Feb 21, 2012 at 7:51 PM, Matthew Weier O'Phinney <[hidden email]> wrote:
   Should we provide auto-escaping by default within the PhpRenderer?



I'm against that.

1. Auto-escaping primary goal is to (implicitly) make XSS attacks harder, but it does not fully prevent them or prevent any other types of attacks.

2. It's bad because it's "implicit", let alone "enabled by default". There should be nothing implicit in the way I echo strings. When I do <?php echo $foo?> I expect it to just echo string. In my understanding it's the same as <?php echo $this->foo?> and there should be no shenanigans going on in the background.

3. Nearly everything in zf2 is _explicit_ , optional, modular and (quite) well separatable. 

4. Auto-escaping has the same bad smell as php 4 magic quotes (deprecated since php 5.3).

5. Escaping in HTML is context-sensitive. htmlentities() is not the same as rawhtmlemtities(), nor addslashes() nor rawurlencode() not mentioning things like strip_tags(). Depending on the fragment I'd want to escape an html argument, or javascript string, or xml entity, or url, or flash var, or whatever else - each of those requires different type of escaping.

6. High-level security concerns should not be general framework's responsibility. It's another story for a CMS or CM Framework (like Drupal or Wordpress) which have different audiences and workflows. ZF2 is low-level, universal and general-purpose app framework so it should never attempt to do anything automagically, let alone fiddle with values.

7. There is no easy way to make it work for nested structures (like echo $this->user->name). This means we cannot advertise this feature as a holy grill of security and XSS prevention, because it will only bring pain to less-experienced users (same argument as with magic quotes being dropped from php)





-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]


 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Artur Bodera
In reply to this post by weierophinney
On Tue, Feb 21, 2012 at 8:36 PM, Matthew Weier O'Phinney <[hidden email]> wrote:
So, is this a vote for "remove auto-escaping" in the current
PhpRenderer, or a vote for a full-on templating engine, ala Twig,
Smarty, etc.?

Or:

 * remove auto-escaping
 * NOT attempt to create (yet another) templating engine
 * introduce out-of-the-box Twig and Smarty renderers


Problem solved.


-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

weierophinney
Administrator
In reply to this post by Tomáš Fejfar
-- Tomáš Fejfar <[hidden email]> wrote
(on Tuesday, 21 February 2012, 08:27 PM +0100):
> The auto-escape implementation in PhpRenderer (as of beta2) is bad and
> misleading and should NOT be there. Same goes to auto-extracting (as nice as it
> in theory sounds) unless there is some middle layer that allows completly
> different syntax then normal PHP variable syntax that cannot be confused for
> local variable...

Let's talk about escaping only at this time; if there are other issues
you want to bring up, do so in a separate thread, and give the _why_
behind your arguments (instead of simply "is bad", which is completely a
subjective statement).

> Content-aware escaping would be a lovely feature and proper caching should be
> an easy opt-in (opt-out even better but I understand it's politically difficult
> to do in library code).

You say these things as if they'd take only a few minutes to code. I
know from experience it's a lot more complex than that. :)


> On Tue, Feb 21, 2012 at 7:51 PM, Matthew Weier O'Phinney <[hidden email]>
> wrote:
>
>     Hey, all!
>
>     Last Friday, I submitted the pull request for the view layer:
>
>        https://github.com/zendframework/zf2/pull/801
>
>     I've had some back and forth with Rob Allen as he reviews it for
>     inclusion, but we're almost ready at this point.
>
>     One lingering question remains, one that isn't directly related to the
>     scope of the RFC, but which I'm interested in answering anyways:
>
>        Should we provide auto-escaping by default within the PhpRenderer?
>
>     The reason I ask is this:
>
>      * In ZF1, you knew that if you didn't escape() a variable, you had
>       better have a good reason.
>      * In ZF2, _SCALAR_ variables will be escaped by default, but variables
>       within arrays or objects are not. So, now you need to know that you
>       need to escape array/object members, but not others.
>
>     As others have pointed out, this second point actually can be a bit
>     harder to explain to users, and may lead to WTF?!?!?! moments when they
>     assume "echo $this->someObject->property" will be escaped, and is not.
>
>     The "solution" to escaping is to:
>
>      * introduce context-specific escaping, which would require parsing and
>       compiling templates, OR
>      * running the final output through a sanitiser of some sort
>
>     Both of which introduce complexity and degrade performance (though
>     performance can be mitigated by good caching, obviously).
>
>     In the meantime, however, my question stands:
>
>        Should we provide auto-escaping by default within the PhpRenderer?
>    
>     --
>     Matthew Weier O'Phinney
>     Project Lead            | [hidden email]
>     Zend Framework          | http://framework.zend.com/
>     PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
>
>

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

weierophinney
Administrator
In reply to this post by Artur Bodera
Artur --

Thanks for your very detailed arguments against auto-escaping! These
are, quite seriously, excellent.


-- Artur Bodera <[hidden email]> wrote
(on Tuesday, 21 February 2012, 08:58 PM +0100):

> On Tue, Feb 21, 2012 at 7:51 PM, Matthew Weier O'Phinney <[hidden email]>
> wrote:
>
>        Should we provide auto-escaping by default within the PhpRenderer?
>
>
>
>
> I'm against that.
>
> 1. Auto-escaping primary goal is to (implicitly) make XSS attacks harder, but
> it does not fully prevent them or prevent any other types of attacks.
>
> 2. It's bad because it's "implicit", let alone "enabled by default". There
> should be nothing implicit in the way I echo strings. When I do <?php echo
> $foo?> I expect it to just echo string. In my understanding it's the same as <?
> php echo $this->foo?> and there should be no shenanigans going on in the
> background.
>
> 3. Nearly everything in zf2 is _explicit_ , optional, modular and (quite) well
> separatable.
>
> 4. Auto-escaping has the same bad smell as php 4 magic quotes (deprecated since
> php 5.3).
>
> 5. Escaping in HTML is context-sensitive. htmlentities() is not the same as
> rawhtmlemtities(), nor addslashes() nor rawurlencode() not mentioning things
> like strip_tags(). Depending on the fragment I'd want to escape an html
> argument, or javascript string, or xml entity, or url, or flash var, or
> whatever else - each of those requires different type of escaping.
>
> 6. High-level security concerns should not be general framework's
> responsibility. It's another story for a CMS or CM Framework (like Drupal or
> Wordpress) which have different audiences and workflows. ZF2 is low-level,
> universal and general-purpose app framework so it should never attempt to do
> anything automagically, let alone fiddle with values.
>
> 7. There is no easy way to make it work for nested structures (like echo
> $this->user->name). This means we cannot advertise this feature as a holy grill
> of security and XSS prevention, because it will only bring pain to
> less-experienced users (same argument as with magic quotes being dropped from
> php)
>
>
>
> ps: http://www.php.net/manual/en/security.magicquotes.whynot.php
>
>
> --
>       __
>      /.)\   +48 695 600 936
>      \(./   [hidden email]
>
>  

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Marc Bennewitz (private)
In reply to this post by weierophinney
Hello Matthew,

I vote for removing auto-escaping because it's not working well on different types of data
on a view layer port who not only developers are working on.
-> You can't explain a designer that "echo $string" is different than "echo $object"

As you noted to have a well auto-escaping we require a parsing mechanism but thats
not part of ZF2 in my opinion - but we can ship with 1 or 2 wrappers for template engines
like a TwigRenderer

Regards,
Marc

On 21.02.2012 19:51, weierophinney [via Zend Framework Community] wrote:
Hey, all!

Last Friday, I submitted the pull request for the view layer:

    https://github.com/zendframework/zf2/pull/801

I've had some back and forth with Rob Allen as he reviews it for
inclusion, but we're almost ready at this point.

One lingering question remains, one that isn't directly related to the
scope of the RFC, but which I'm interested in answering anyways:

    Should we provide auto-escaping by default within the PhpRenderer?

The reason I ask is this:

 * In ZF1, you knew that if you didn't escape() a variable, you had
   better have a good reason.
 * In ZF2, _SCALAR_ variables will be escaped by default, but variables
   within arrays or objects are not. So, now you need to know that you
   need to escape array/object members, but not others.

As others have pointed out, this second point actually can be a bit
harder to explain to users, and may lead to WTF?!?!?! moments when they
assume "echo $this->someObject->property" will be escaped, and is not.

The "solution" to escaping is to:

 * introduce context-specific escaping, which would require parsing and
   compiling templates, OR
 * running the final output through a sanitiser of some sort

Both of which introduce complexity and degrade performance (though
performance can be mitigated by good caching, obviously).

In the meantime, however, my question stands:

    Should we provide auto-escaping by default within the PhpRenderer?

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc



If you reply to this email, your message will be added to the discussion below:
http://zend-framework-community.634137.n4.nabble.com/View-Layer-lingering-questions-tp4407918p4407918.html
To start a new topic under ZF Contributor, email [hidden email]
To unsubscribe from ZF Contributor, click here.
NAML
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Pádraic Brady
Artur brings up a good point that one of the flaws with auto-escaping
is that the default will always be html escaping - which is not
applicable to other contexts like javascript. Even with auto escaping,
users would still need to create escape() like functions for those -
and I have a feeling very few actually do this at the moment. Either
way, it's well worth considering putting a bit more effort into
supporting multiple contexts and including more robust escaping
mechanisms for those who wouldn't mind exchanging a small performance
hit for far tighter escaping security. That alone is something not
currently typical of any framework but it could offer more value than
even the convenience of autoescaping could.

If we want to go down the route of offering a template system (other
than one in pure PHP), we can wait for ZF 3.0 to deliver such an
option - I don't think it's a trivial task we could pull off well that
quickly and there are already enough mature options out there that
reinventing that wheel shouldn't be necessary. It's still possible
that parsing PHP templates could work for auto escaping - PHP is
certainly easier to parse than HTML - which is basically designed in a
way that makes parsing difficult. That still wouldn't solve limiting
the autoescaping to only those contexts for which it's valid. That
would require parsing the HTML itself to determine the context, prior
to setting the escaping method in the compiled cacheable version -
which is probably doable with a bit of work.

Question is which option is likely to not introduce delays into releasing ZF2?

Paddy

On Tue, Feb 21, 2012 at 9:26 PM, Marc Bennewitz <[hidden email]> wrote:

> Hello Matthew,
>
> I vote for removing auto-escaping because it's not working well on different
> types of data
> on a view layer port who not only developers are working on.
> -> You can't explain a designer that "echo $string" is different than "echo
> $object"
>
> As you noted to have a well auto-escaping we require a parsing mechanism but
> thats
> not part of ZF2 in my opinion - but we can ship with 1 or 2 wrappers for
> template engines
> like a TwigRenderer
>
> Regards,
> Marc
>
>
> On 21.02.2012 19:51, weierophinney [via Zend Framework Community] wrote:
>
> Hey, all!
>
> Last Friday, I submitted the pull request for the view layer:
>
>     https://github.com/zendframework/zf2/pull/801
>
> I've had some back and forth with Rob Allen as he reviews it for
> inclusion, but we're almost ready at this point.
>
> One lingering question remains, one that isn't directly related to the
> scope of the RFC, but which I'm interested in answering anyways:
>
>     Should we provide auto-escaping by default within the PhpRenderer?
>
> The reason I ask is this:
>
>  * In ZF1, you knew that if you didn't escape() a variable, you had
>    better have a good reason.
>  * In ZF2, _SCALAR_ variables will be escaped by default, but variables
>    within arrays or objects are not. So, now you need to know that you
>    need to escape array/object members, but not others.
>
> As others have pointed out, this second point actually can be a bit
> harder to explain to users, and may lead to WTF?!?!?! moments when they
> assume "echo $this->someObject->property" will be escaped, and is not.
>
> The "solution" to escaping is to:
>
>  * introduce context-specific escaping, which would require parsing and
>    compiling templates, OR
>  * running the final output through a sanitiser of some sort
>
> Both of which introduce complexity and degrade performance (though
> performance can be mitigated by good caching, obviously).
>
> In the meantime, however, my question stands:
>
>     Should we provide auto-escaping by default within the PhpRenderer?
>
> --
> Matthew Weier O'Phinney
> Project Lead            | [hidden email]
> Zend Framework          | http://framework.zend.com/
> PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://zend-framework-community.634137.n4.nabble.com/View-Layer-lingering-questions-tp4407918p4407918.html
> To start a new topic under ZF Contributor, email
> [hidden email]
> To unsubscribe from ZF Contributor, click here.
> NAML



--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

elydelacruz
This post has NOT been accepted by the mailing list yet.
In reply to this post by weierophinney
I vote nay for auto-escaping of vars passed to the view in zf2. The user should have the option to choose what he wants and besides as long as the opt-in /(couple/de-couple) solutions are advertised in the refman/docs users are more likely to opt-in or implement a solution to take care of these concerns. Padraic's Idea of having different context solutions for escaping different kinds of vars for the view sounds cool (allows user to use solution on an "as needed basis"). Padraic's idea also coincides with some of Artur Bodera's ideas:
3. Nearly everything in zf2 is _explicit_ , optional, modular and (quite) well separatable.

Also, I partially agree with Bodera when he says:

6. High-level security concerns should not be general framework's responsibility.

This statement is only partially true though because zf deals with both low and high level constructs. Though Bodera might have been hinting at something else here. Maybe that the framework should be flexible in this respect and not make it it's responsibility, solely, in this respect.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Artur Bodera
In reply to this post by Pádraic Brady
On Tue, Feb 21, 2012 at 11:11 PM, Pádraic Brady <[hidden email]> wrote:
Artur brings up a good point that one of the flaws with auto-escaping
is that the default will always be html escaping - which is not
applicable to other contexts like javascript. Even with auto escaping,
users would still need to create escape() like functions for those -
and I have a feeling very few actually do this at the moment. Either
way, it's well worth considering putting a bit more effort into
supporting multiple contexts and including more robust escaping
mechanisms for those who wouldn't mind exchanging a small performance
hit for far tighter escaping security. That alone is something not
currently typical of any framework but it could offer more value than
even the convenience of autoescaping could.

If we want to go down the route of offering a template system (other
than one in pure PHP), we can wait for ZF 3.0 to deliver such an
option - I don't think it's a trivial task we could pull off well that
quickly and there are already enough mature options out there that
reinventing that wheel shouldn't be necessary. It's still possible
that parsing PHP templates could work for auto escaping - PHP is
certainly easier to parse than HTML - which is basically designed in a
way that makes parsing difficult. That still wouldn't solve limiting
the autoescaping to only those contexts for which it's valid. That
would require parsing the HTML itself to determine the context, prior
to setting the escaping method in the compiled cacheable version -
which is probably doable with a bit of work.

Question is which option is likely to not introduce delays into releasing ZF2?


Paddy, there's a reason for general-purpose frameworks not shipping that by default.

It can be easily read from template engines' manual pages. Template engines for PHP were always an oxymoron,  because PHP is a template engine by its very definition. The sole purpose of Smarty 1.0 was to make it easier for non-programmers (read: web developers, layout/template designers, freelance helpers etc.) to help erect web sites without the risk of compromising application data or breaking (too much) stuff.

If you call a proper php template engine (like Smarty)  _only_ a wrapper for PHP, that would be hurtful to the hundreds of man-hours put to create them. There's a lot of thought put into them to  isolate the design from code (designers from coders), so that these two groups of people could co-exist without breaking the application with each and every commit.

That means, that  {{ foo.property }} in Twig is NOT ONLY escaping and echoing $foo->property, but actually performing a myriad of others tasks:

(see "implementation"here:)


Because PhpRenderer exposes _full_, bare PHP it's impossible (or not feasible) to secure it. It's also quite pointless, because "PHP IS A TEMPLATING LANGUAGE" which many will point out on the Internet. However, it's not suitable for non-programmers for several reasons (security being one of them). I am aware, that there are a few php-language based php templating engines which try to "sandbox" and "filter" php code (i.e. http://phpsavant.com/) , but that's like buying a Lamborgini and going offroading with it.

From my experience, templating with PHP is used in this fashion:
  1. If the application is developed by a (small) group of programmers, it usually uses raw php (PhpRendererer) to generate HTML.
  2. If the application is developer by a group of programmers mixed with designers, it uses meta-language templating engine (like Twig).
  3. If the application is huge, it uses one of the major (most popular) templating languages or XML/XSLT.

It's usually assumed that:
  • Programmers know what they're doing.
  • Designers(artists) break stuff constantly and have to be kept away from the application (in cages preferably)

btw:
(sorry, just found those ;) 

How many different escaping options does Smarty provide?

... but the most popular templating engine refuses to turn auto-escaping on by default:

... it does however have a facility for that:

.. but still, it's not part of the default "security measures"


-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

akrabat
In reply to this post by weierophinney
Agreed.

Based on what I've read in this thread, I think we should take out the auto-escaping too. It would be good to be able to use overridden ViewModels everywhere in the view-layer though so that if someone wanted to do something similar, they could do easily.

I do think that expecting front end HTML-focussed developers to remember to escape variables in the view scripts is optimistic and I'd love to know the average number of escaping flaws per view script in currently shipping ZF1 apps! I bet its much higher than contributors to this list are willing to believe as we are, by the virtue of just being here, above average when it comes to understanding the problem and being aware of it.

I really see no reason whatsoever  to develop our own template system, ever.

In an ideal world, I'd like to see us ship working out-of-the-box integration with a non-PHP-based view template system. I don't mind which one, but would suggest that Mustache[1] looks like a reasonable choice as it's not trying to be a "safe version of PHP" and is small. Also, I think the ViewModel changes allowing closures to be added as properties should make Mustache support relatively easy and validate that our view-layer really is template-system agnostic. It  would also force us to write a form system that isn't php-view-script dependent :)  


Regards,

Rob...


On 21 Feb 2012, at 21:25, Matthew Weier O'Phinney wrote:

> Artur --
>
> Thanks for your very detailed arguments against auto-escaping! These
> are, quite seriously, excellent.
>


[1] https://github.com/weierophinney/phly_mustache#readme

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Pádraic Brady
In reply to this post by Artur Bodera
> Paddy, there's a reason for general-purpose frameworks not shipping that by
> default.

Because it's non-trivial to implement and HTML was authored in Hell? ;)

> It can be easily read from template engines' manual pages. Template engines
> for PHP were always an oxymoron,  because PHP is a template engine by its
> very definition. The sole purpose of Smarty 1.0 was to make it easier for
> non-programmers (read: web developers, layout/template designers, freelance
> helpers etc.) to help erect web sites without the risk of compromising
> application data or breaking (too much) stuff.
>
> If you call a proper php template engine (like Smarty)  _only_ a wrapper for
> PHP, that would be hurtful to the hundreds of man-hours put to create them.
> There's a lot of thought put into them to  isolate the design from code
> (designers from coders), so that these two groups of people could co-exist
> without breaking the application with each and every commit.

You're preaching to the choir…

> That means, that  {{ foo.property }} in Twig is NOT ONLY escaping and
> echoing $foo->property, but actually performing a myriad of others tasks:
>
> (see "implementation"here:)
> http://twig.sensiolabs.org/doc/templates.html#variables

Not to be too picky, but that's a good example of why I strongly
prefer PHP templating over something like Twig.

> Because PhpRenderer exposes _full_, bare PHP it's impossible (or not
> feasible) to secure it. It's also quite pointless, because "PHP IS A
> TEMPLATING LANGUAGE" which many will point out on the Internet. However,
> it's not suitable for non-programmers for several reasons (security being
> one of them). I am aware, that there are a few php-language based php
> templating engines which try to "sandbox" and "filter" php code
> (i.e. http://phpsavant.com/) , but that's like buying a Lamborgini and going
> offroading with it.

Impossible or not feasible is overdoing it. PHP parses HTML documents
all the time, can somehow figure out where the PHP is, and manages to
execute it without overextending it's reach. Figuring out what is PHP
or not is not difficult in templates. Parsing PHP itself is about the
simplest task imaginable. Both are indicative that autoescaping a pure
PHP template should be very possible - if you could manage to detect
the right context for it.

> From my experience, templating with PHP is used in this fashion:
>
> If the application is developed by a (small) group of programmers, it
> usually uses raw php (PhpRendererer) to generate HTML.
> If the application is developer by a group of programmers mixed with
> designers, it uses meta-language templating engine (like Twig).
> If the application is huge, it uses one of the major (most popular)
> templating languages or XML/XSLT.

Similar experience here - we do however impress the need for PHP
templates a bit further which our designers haven't (so far)
complained about. On very large projects, using a non-PHP solution
will start making sense since there's a lot more code, a lot less
experienced programmers (too many cooks…) and it's far easier for
mistakes to go unnoticed. A lot of that, however, is a function of
keeping PHP templating simple - you don't see a lot of people
compiling PHP into PHP ;).

> It's usually assumed that:
>
> Programmers know what they're doing.
> Designers(artists) break stuff constantly and have to be kept away from the
> application (in cages preferably)

Doesn't always work that way - then again, maybe our designers are
exceptionally weird. ;) They eat PHP and Ruby for breakfast so we get
very good mileage out of them.

> btw:
> Paddy on
> topic: http://blog.astrumfutura.com/2009/10/is-php-a-worthy-template-language-well-of-course-it-is/
> MWoP on topic: http://mwop.net/blog/2-PHP-and-Template-Engines
> (sorry, just found those ;)
>
> How many different escaping options does Smarty provide?
> http://www.smarty.net/docsv2/en/language.modifier.escape
>
> ... but the most popular templating engine refuses to turn auto-escaping on
> by default:
> http://www.smarty.net/forums/viewtopic.php?t=7684&sid=28456f7bef2d37660d16cc461bed2957
>
> ... it does however have a facility for that:
> http://www.smarty.net/docs/en/variable.default.modifiers.tpl
>
> .. but still, it's not part of the default "security measures"
> http://www.smarty.net/docs/en/advanced.features.tpl#advanced.features.security

Thing is, auto escaping works very nicely once you understand it's
limitations. I've used a homegrown setup (pre-ZF adoption) since
around 2007 or so which I really liked. For ZF2, we can't make the
same assumption that users will understand those limitations and under
those circumstances, the limitations may well be too confusing to be
worth it. All I outlined in my options, what that it could be possible
- but only via a compilation stage rather than something we could add
programmatically.

Paddy


--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Artur Bodera
On Wed, Feb 22, 2012 at 12:59 PM, Pádraic Brady <[hidden email]> wrote:
All I outlined in my options, what that it could be possible
- but only via a compilation stage rather than something we could add
programmatically.

... which boils down to compiling PHP to PHP that you just bashed ;)

... or creating another templating language, which is IMHO a waste of time.

And context sniffing sounds to me like implementing a CrystallBall algorithm ... trying to make code that's smarter than coder :)


btw: come again? what do you feed your designers ? :-) 

A.

-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Artur Bodera
In reply to this post by Pádraic Brady
On Wed, Feb 22, 2012 at 12:59 PM, Pádraic Brady <[hidden email]> wrote:
Impossible or not feasible is overdoing it. PHP parses HTML documents
all the time, can somehow figure out where the PHP is, and manages to
execute it without overextending it's reach.

Actually, it's dumb as a shoe. It relies on the existence of (balanced) 
<?php ?> tags. Miss one of those, and it'll just WSOD. I wouldn't call it
"managing the process"

 
Figuring out what is PHP
or not is not difficult in templates. Parsing PHP itself is about the
simplest task imaginable.

You're mistaken here...
 
Both are indicative that autoescaping a pure
PHP template should be very possible - if you could manage to detect
the right context for it.

Well, I'd recommend reading Twig and Smarty documentation sections
on security and "inline PHP code". That's where i.e. Smarty allows a 
designer (ugh) to inject a few lines of PHP code directly into smarty template.

So now it has to do double-parsing, detect blocks of PHP, wrap around it,
then parse the rest of smarty tags and spew out a compiled PHP which
is then parsed again by Zend engine.

It's a nightmare because for all those benefits of a parsed templating engine
that automagically takes care of everything (including escaping for security)
are now diminished ... and a theoretically safe text-only smarty template 
is polluted with executable (dangerous) php code ... eval() anyone?

This discussion is like the history of magic_quotes repeating itself... tens of
bugs filed on http://bugs.php.net where novice developers cry that:

1) Their contact forms send emails with \"\""\" in subject lines
2) Their contact forms send proper emails but someone hax0r3d their site.


-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]








-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: View Layer: lingering questions

Pádraic Brady
In reply to this post by Artur Bodera
Now, now, I never bashed compiling PHP into PHP. I simply noted that PHP  template systems don't do it in order to remain simple ;).

Paddy

On 22 Feb 2012, at 12:43,  Bodera <[hidden email]> wrote:
 
On Wed, Feb 22, 2012 at 12:59 PM, Pádraic Brady <[hidden email]> wrote:
All I outlined in my options, what that it could be possible
- but only via a compilation stage rather than something we could add
programmatically.

... which boils down to compiling PHP to PHP that you just bashed ;)

... or creating another templating language, which is IMHO a waste of time.

And context sniffing sounds to me like implementing a CrystallBall algorithm ... trying to make code that's smarter than coder :)


btw: come again? what do you feed your designers ? :-) 

A.

-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]




 
12
Loading...