Performance ZF1 vs. ZF2

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Performance ZF1 vs. ZF2

Ralf Eggert
Hi,

I wonder if anyone has made a performance comparison between ZF1 and ZF2
recently. The newest one I found is from February 2012.

http://www.enrise.com/2012/02/zend-framework-2-performance/

Is ZF2 still slower than ZF1?

Regards,

Ralf
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

EvanDotPro
Hi Ralf,

On Tue, Oct 23, 2012 at 12:24 PM, Ralf Eggert <[hidden email]> wrote:
> Hi,
>
> I wonder if anyone has made a performance comparison between ZF1 and ZF2
> recently. The newest one I found is from February 2012.
>
> http://www.enrise.com/2012/02/zend-framework-2-performance/
>
> Is ZF2 still slower than ZF1?

That question can't really be answered in the way you're looking for,
unfortunately. While the hello _might_ be slower in ZF2 than ZF1 (that
may not be true anymore, I've seen mixed reports and haven't ran an
up-to-date benchmark myself yet), the architecture in ZF2 allows for
more optimized application bootstrapping so only the resources needed
for a request are initialized.

That said, 99.999% of the time, framework performance is not the
bottle-neck that applications run into, regardless of what the
baseline performance. Generally there's more productive places to
optimize performance such as caching, database indexes/queries/etc.
I'd say at this point ZF2 performance is very much within the
"acceptable" range as far as base-line performance goes for
medium-to-large-to-very-large applications. We have an SMS gateway
platform we're refactoring to ZF2 that handles millions of uncached
requests per day, and tests so far show that ZF2 performance will not
be a problem.

--
Evan Coury
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Ralf Eggert
Hi Evan,

thanks for your thoughts. Well, you know that, I know that, and maybe
most of the others in this list know that too.

But out there in the wild, people are googling for performance reports
and what they find is that ZF2 is five times slower than ZF1. And much
slower than all the rest. We can argue here as much as we want, but some
people will not take a closer look on the ZF2 when they have the feeling
that it is slower than ZF1.

Since the stable ZF 2.0.0 was released I got almost a dozen mails from
users which are afraid of switching to ZF2 since it is said to be much
slower than ZF1.

I am not a performance tracking expert, but maybe we could set something
up to compare at least ZF1 with ZF2 on a kind-of real world application?

Regards,

Ralf
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Marco Pivetta
That would be really useful, Ralf. Nobody has ever done a decent comparison yet, and everybody is using the skeleton app for benchmarks, which:

 1) does not use the dumped composer classmap
 2) uses a whole lot of view helpers, which eat up a lot of performance
 3) has not a classmap (provided by EdpSuperluminal)

I'd also suggest using Siege instead of ApacheBench, since AB reports fancy stuff when working with "Keep-Alive".
Marco Pivetta

http://twitter.com/Ocramius     

http://ocramius.github.com/



On 24 October 2012 09:27, Ralf Eggert <[hidden email]> wrote:
Hi Evan,

thanks for your thoughts. Well, you know that, I know that, and maybe
most of the others in this list know that too.

But out there in the wild, people are googling for performance reports
and what they find is that ZF2 is five times slower than ZF1. And much
slower than all the rest. We can argue here as much as we want, but some
people will not take a closer look on the ZF2 when they have the feeling
that it is slower than ZF1.

Since the stable ZF 2.0.0 was released I got almost a dozen mails from
users which are afraid of switching to ZF2 since it is said to be much
slower than ZF1.

I am not a performance tracking expert, but maybe we could set something
up to compare at least ZF1 with ZF2 on a kind-of real world application?

Regards,

Ralf

Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Tomáš Fejfar
It's still a good idea to keep an eye on the speed. 

What about keeping an akrabat app for 1.12 and each 2.X version somewhere (somewhere zend-internally - to make sure the testing conditions are coherent) and siege it each time to make sure the performance doesn't change (or does change for the better). 

I've already suggested this once in the past, but it was not accepted. It needs to be done on some reasonably stable enviroment (probably some zend-internal server/EC2) - a dev machine is no good for such long-therm comparison as it's performance may vary in time based on utilities, system state, etc... that's why I don't want to take care of that personally as I don't have access to such system. 

On Wed, Oct 24, 2012 at 9:52 AM, Marco Pivetta <[hidden email]> wrote:
That would be really useful, Ralf. Nobody has ever done a decent comparison yet, and everybody is using the skeleton app for benchmarks, which:

 1) does not use the dumped composer classmap
 2) uses a whole lot of view helpers, which eat up a lot of performance
 3) has not a classmap (provided by EdpSuperluminal)

I'd also suggest using Siege instead of ApacheBench, since AB reports fancy stuff when working with "Keep-Alive".
Marco Pivetta

http://twitter.com/Ocramius     

http://ocramius.github.com/




On 24 October 2012 09:27, Ralf Eggert <[hidden email]> wrote:
Hi Evan,

thanks for your thoughts. Well, you know that, I know that, and maybe
most of the others in this list know that too.

But out there in the wild, people are googling for performance reports
and what they find is that ZF2 is five times slower than ZF1. And much
slower than all the rest. We can argue here as much as we want, but some
people will not take a closer look on the ZF2 when they have the feeling
that it is slower than ZF1.

Since the stable ZF 2.0.0 was released I got almost a dozen mails from
users which are afraid of switching to ZF2 since it is said to be much
slower than ZF1.

I am not a performance tracking expert, but maybe we could set something
up to compare at least ZF1 with ZF2 on a kind-of real world application?

Regards,

Ralf


Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

till
In reply to this post by Marco Pivetta
Maybe someone could re-do the skeleton application? Performance is important. E.g. throwing a lot of hardware at it is something we usually try to avoid. :-)

Till

On Wednesday, October 24, 2012 at 9:52 AM, Marco Pivetta wrote:

> That would be really useful, Ralf. Nobody has ever done a decent comparison yet, and everybody is using the skeleton app for benchmarks, which:
>
> 1) does not use the dumped composer classmap
> 2) uses a whole lot of view helpers, which eat up a lot of performance
> 3) has not a classmap (provided by EdpSuperluminal)
>
> I'd also suggest using Siege instead of ApacheBench, since AB reports fancy stuff when working with "Keep-Alive".
> Marco Pivetta
>
> http://twitter.com/Ocramius 
>
> http://ocramius.github.com/
>
>
>
> On 24 October 2012 09:27, Ralf Eggert <[hidden email] (mailto:[hidden email])> wrote:
> > Hi Evan,
> >
> > thanks for your thoughts. Well, you know that, I know that, and maybe
> > most of the others in this list know that too.
> >
> > But out there in the wild, people are googling for performance reports
> > and what they find is that ZF2 is five times slower than ZF1. And much
> > slower than all the rest. We can argue here as much as we want, but some
> > people will not take a closer look on the ZF2 when they have the feeling
> > that it is slower than ZF1.
> >
> > Since the stable ZF 2.0.0 was released I got almost a dozen mails from
> > users which are afraid of switching to ZF2 since it is said to be much
> > slower than ZF1.
> >
> > I am not a performance tracking expert, but maybe we could set something
> > up to compare at least ZF1 with ZF2 on a kind-of real world application?
> >
> > Regards,
> >
> > Ralf


Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Marco Pivetta
@till the only thing that can be avoided in the skeleton app is all those view helpers. Otherwise it's just fine as it is.

Marco Pivetta

http://twitter.com/Ocramius     

http://ocramius.github.com/



On 24 October 2012 16:07, till <[hidden email]> wrote:
Maybe someone could re-do the skeleton application? Performance is important. E.g. throwing a lot of hardware at it is something we usually try to avoid. :-)

Till

On Wednesday, October 24, 2012 at 9:52 AM, Marco Pivetta wrote:

> That would be really useful, Ralf. Nobody has ever done a decent comparison yet, and everybody is using the skeleton app for benchmarks, which:
>
> 1) does not use the dumped composer classmap
> 2) uses a whole lot of view helpers, which eat up a lot of performance
> 3) has not a classmap (provided by EdpSuperluminal)
>
> I'd also suggest using Siege instead of ApacheBench, since AB reports fancy stuff when working with "Keep-Alive".
> Marco Pivetta
>
> http://twitter.com/Ocramius
>
> http://ocramius.github.com/
>
>
>
> On 24 October 2012 09:27, Ralf Eggert <[hidden email] (mailto:[hidden email])> wrote:
> > Hi Evan,
> >
> > thanks for your thoughts. Well, you know that, I know that, and maybe
> > most of the others in this list know that too.
> >
> > But out there in the wild, people are googling for performance reports
> > and what they find is that ZF2 is five times slower than ZF1. And much
> > slower than all the rest. We can argue here as much as we want, but some
> > people will not take a closer look on the ZF2 when they have the feeling
> > that it is slower than ZF1.
> >
> > Since the stable ZF 2.0.0 was released I got almost a dozen mails from
> > users which are afraid of switching to ZF2 since it is said to be much
> > slower than ZF1.
> >
> > I am not a performance tracking expert, but maybe we could set something
> > up to compare at least ZF1 with ZF2 on a kind-of real world application?
> >
> > Regards,
> >
> > Ralf



Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

EvanDotPro
In reply to this post by Ralf Eggert

Hi Ralf,

On Wed, Oct 24, 2012 at 12:27 AM, Ralf Eggert <[hidden email]> wrote:
> Hi Evan,
>
> thanks for your thoughts. Well, you know that, I know that, and maybe
> most of the others in this list know that too.
>
> But out there in the wild, people are googling for performance reports
> and what they find is that ZF2 is five times slower than ZF1. And much
> slower than all the rest. We can argue here as much as we want, but some
> people will not take a closer look on the ZF2 when they have the feeling
> that it is slower than ZF1.

Yeah -- sadly this is true. With the current benchmarks in the wild, we definitely have a marketing issue; and yes, it's a marketing issue.

We've talked about performance quite a bit. We know that there is room for improvement and will gladly accept and performance-related PR's. I've done quite a few PR's in late betas and during the RC stages to squeeze a bit more of performance in some of the obvious places. Unfortunately, it starts to get tricky beyond a certain point once each method call represents no more than 1% or so of the execution time.

--
Evan Coury

Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Stewart Lord
In reply to this post by Ralf Eggert
 > But out there in the wild, people are googling for performance reports
 > and what they find is that ZF2 is five times slower than ZF1.

Don't forget about the problem with ab and siege as discussed in the
"Performance Oddity" thread. There is some weirdness around headers and
keep-alive that are dragging ZF2's numbers way down.

Stew


On 12-10-24 12:27 AM, Ralf Eggert wrote:

> Hi Evan,
>
> thanks for your thoughts. Well, you know that, I know that, and maybe
> most of the others in this list know that too.
>
> But out there in the wild, people are googling for performance reports
> and what they find is that ZF2 is five times slower than ZF1. And much
> slower than all the rest. We can argue here as much as we want, but some
> people will not take a closer look on the ZF2 when they have the feeling
> that it is slower than ZF1.
>
> Since the stable ZF 2.0.0 was released I got almost a dozen mails from
> users which are afraid of switching to ZF2 since it is said to be much
> slower than ZF1.
>
> I am not a performance tracking expert, but maybe we could set something
> up to compare at least ZF1 with ZF2 on a kind-of real world application?
>
> Regards,
>
> Ralf
>
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

weierophinney
Administrator
In reply to this post by Ralf Eggert
-- Ralf Eggert <[hidden email]> wrote
(on Wednesday, 24 October 2012, 09:27 AM +0200):
> thanks for your thoughts. Well, you know that, I know that, and maybe
> most of the others in this list know that too.
>
> But out there in the wild, people are googling for performance reports
> and what they find is that ZF2 is five times slower than ZF1. And much
> slower than all the rest. We can argue here as much as we want, but some
> people will not take a closer look on the ZF2 when they have the feeling
> that it is slower than ZF1.

Ralph is working on benchmarks as one of his priorities starting this
week. The plan is to develop a non-trivial app (likely based off the
tutorial application), and have the benchmarking script hit several
different end-points. We will re-create the app in both ZF1 and ZF2 so
as to see the differences in performance.

The benchmarks that exist are, quite frankly, old, and none of them test
the stable version. If you see folks worried about that, point out that
fact. We should have something to report in the next couple of weeks.

> Since the stable ZF 2.0.0 was released I got almost a dozen mails from
> users which are afraid of switching to ZF2 since it is said to be much
> slower than ZF1.
>
> I am not a performance tracking expert, but maybe we could set something
> up to compare at least ZF1 with ZF2 on a kind-of real world application?
>
> Regards,
>
> Ralf
>

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
PGP key: http://framework.zend.com/zf-matthew-pgp-key.asc
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Ralf Eggert
Hi Matthew,

hope the ZendCon went well.

> Ralph is working on benchmarks as one of his priorities starting this
> week. The plan is to develop a non-trivial app (likely based off the
> tutorial application), and have the benchmarking script hit several
> different end-points. We will re-create the app in both ZF1 and ZF2 so
> as to see the differences in performance.
>
> The benchmarks that exist are, quite frankly, old, and none of them test
> the stable version. If you see folks worried about that, point out that
> fact. We should have something to report in the next couple of weeks.

Great news!

The problem so far is not with the people who come up with this and
worry about it. Its easy to tell them the facts. The bigger problem is
with the people who google the comparison results and stop taking ZF2 in
account once they read about the slow ZF2.

I am looking forward to the benchmarks.

Regards,

Ralf
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Tomáš Fejfar
In reply to this post by weierophinney
Glad to hear that!

Than just google-bomb the result for KWs like "zf2 speed" and we're done :D

On Mon, Oct 29, 2012 at 4:27 PM, Matthew Weier O'Phinney <[hidden email]> wrote:
Ralph is working on benchmarks as one of his priorities starting this
week. The plan is to develop a non-trivial app (likely based off the
tutorial application), and have the benchmarking script hit several
different end-points. We will re-create the app in both ZF1 and ZF2 so
as to see the differences in performance.

Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

travis
This post has NOT been accepted by the mailing list yet.
Has any progress been made on this front?

I'm anticipating having the same kinds of trouble that Ralf, I think, was mentioning: I'm worried about people not wanting to take a closer look at ZF2 because of how slow they believe it to be.

There are some projects I'm working on that can benefit a great deal from ZF2, but one cannot sacrifice (very much) performance.

My Googling hasn't yielded, other than anecdotal comments, anything positive about ZF2 and performance.  This is not good.  :-/

Thanks...
Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

Marco Pivetta
This post has NOT been accepted by the mailing list yet.
I've been stuck with https://github.com/TechEmpower/FrameworkBenchmarks/pull/218 to get that done. Didn't have time to complete it though.



On Tue, Sep 10, 2013 at 8:49 AM, travis [via Zend Framework Community] <[hidden email]> wrote:
Has any progress been made on this front?

I'm anticipating having the same kinds of trouble that Ralf, I think, was mentioning: I'm worried about people not wanting to take a closer look at ZF2 because of how slow they believe it to be.

There are some projects I'm working on that can benefit a great deal from ZF2, but one cannot sacrifice (very much) performance.

My Googling hasn't yielded, other than anecdotal comments, anything positive about ZF2 and performance.  This is not good.  :-/

Thanks...


If you reply to this email, your message will be added to the discussion below:
http://zend-framework-community.634137.n4.nabble.com/Performance-ZF1-vs-ZF2-tp4657746p4660945.html
To start a new topic under ZF Contributor, email [hidden email]
To unsubscribe from ZF Contributor, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Performance ZF1 vs. ZF2

travis
This post has NOT been accepted by the mailing list yet.
@Marco: Thanks for the update...and the work on trying to get ZF2 added to TechEmpower's list of benchmarked frameworks.