Design/performance question: many modules in a ZF2 application?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Design/performance question: many modules in a ZF2 application?

demiankatz

My application is large and could easily be broken into many components (theme system, configuration manager, custom HTTP service, etc. – probably at least a dozen, maybe even 20-30 parts).  It seems that there are two ways to do this:

 

1.)    Build a single module and separate components through namespacing

2.)    Build a separate module for each component plus an application module that configures all components to make the full application

 

Option #1 seems to have less overhead, but option #2 is attractive since it encourages modularity and reusable design.

 

I have two concerns about #2, though:

 

a.)    Performance: Using large numbers of modules forces a lot of file reading and configuration merging even if certain modules are not used by the current request; i.e. overusing the module system seems like it could cancel out some of the benefits of on-demand loading.

b.)    Namespacing: You can nest modules within a single namespace, but it’s awkward (especially with regard to autoloading).  So rather than having a VuFind namespace, I end up with VuFindComponent1, VuFindComponent2, …, VuFindApplication, etc.  It works, but it’s not pretty.

 

Has anyone gone down this road already?  Has any benchmarking been done to see how well modules scale?  Any advice on finding the right balance between clean design and good performance is much appreciated.

 

thanks,

Demian

Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

Marco Pivetta
Hi there,



On 31 October 2012 14:19, Demian Katz <[hidden email]> wrote:

My application is large and could easily be broken into many components (theme system, configuration manager, custom HTTP service, etc. – probably at least a dozen, maybe even 20-30 parts).  It seems that there are two ways to do this:

 

1.)    Build a single module and separate components through namespacing

Should already be like that
 

2.)    Build a separate module for each component plus an application module that configures all components to make the full application

Only if you can make them independent and re-usable
 

 

Option #1 seems to have less overhead, but option #2 is attractive since it encourages modularity and reusable design.

 

I have two concerns about #2, though:

 

a.)    Performance: Using large numbers of modules forces a lot of file reading and configuration merging even if certain modules are not used by the current request; i.e. overusing the module system seems like it could cancel out some of the benefits of on-demand loading.

File loading is not an issue. Classmaps and APC handle this for you. Modules should be about pluggable functionality though. If your components depend on each other strictly then that's your "functionality". Split it if you provide services you could want to have multiple times and in separate locations

b.)    Namespacing: You can nest modules within a single namespace, but it’s awkward (especially with regard to autoloading).  So rather than having a VuFind namespace, I end up with VuFindComponent1, VuFindComponent2, …, VuFindApplication, etc.  It works, but it’s not pretty.

Not really an advantage, that should happen anyway, regardless of modules or not.

 

Has anyone gone down this road already?  Has any benchmarking been done to see how well modules scale?  Any advice on finding the right balance between clean design and good performance is much appreciated.

 

thanks,

Demian


Also, FYI, merged config is cacheable

Marco Pivetta

http://twitter.com/Ocramius     

http://ocramius.github.com/
Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

EvanDotPro
In reply to this post by demiankatz
Hi,

Don't worry about module loading performance. Run a benchmark -- it's a very lightweight process.

As Marco said, modular design is about re-usability, so split your applications into modules where it makes sense for re-use.

PS: You do not need to build a "theme system" in ZF2 -- modules already "just work" as themes.

-Evan
Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

Jurian Sluiman
In reply to this post by demiankatz
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

RE: Design/performance question: many modules in a ZF2 application?

demiankatz
In reply to this post by EvanDotPro

Thanks (to you, and everyone else who replied) for the feedback – this is very helpful.

 

Regarding my “theme system,” this is a mechanism for creating multiple named sets of templates (that can optionally inherit from one another) to allow multiple themes within an application.  It also has features like automatic switching based on mobile device detection.  I don’t think these things are native framework features, though if I’m reinventing the wheel, please point me in the right direction.

 

thanks,

Demian

 

From: Evan Coury [mailto:[hidden email]]
Sent: Wednesday, October 31, 2012 8:07 PM
To: Demian Katz
Cc: Zend Framework Contributors
Subject: Re: [zf-contributors] Design/performance question: many modules in a ZF2 application?

 

Hi,

 

Don't worry about module loading performance. Run a benchmark -- it's a very lightweight process.

 

As Marco said, modular design is about re-usability, so split your applications into modules where it makes sense for re-use.

 

PS: You do not need to build a "theme system" in ZF2 -- modules already "just work" as themes.

 

-Evan

Reply | Threaded
Open this post in threaded view
|

RE: Design/performance question: many modules in a ZF2 application?

demiankatz
In reply to this post by Jurian Sluiman

Thanks for this – very encouraging.

 

Just one question: does your configuration rely on none of the modules defining autoloading rules for the root \Soflomo namespace?  Or is there a way to convince the autoloader to load \Soflomo\Log from one place, \Soflomo\Mail from another, and fail back to a third location for remaining \Soflomo classes?  I tried to figure this out a few months ago and failed, but maybe I wasn’t trying hard enough.

 

- Demian

 

The only things you must keep in mind are:

  1. If you link the module to your ./module/ dir, make sure you have the module in ./module/Soflomo/Log and not ./module/<name it here> because the autoloader can't find it. We use composer, so that doesn't bother us.
  2. The autoloader config in the module class can't be the simple __NAMESPACE__ => __DIR__ . '/src/' . __NAMESPACE__, if you (like we have) put a /src/Log/ in your module and not a /src/Soflomo/Log/
  3. As point #2, same holds for the composer.json.

As you notice, they are not real big issues, just some tiny parts which deviate from a standard configuration.

Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

Michael Gooden
I cannot confirm this 100%, as I do not have the code on me, but last time I tried this, it worked fine, as long as you had the more specific namespaces earlier in the autoloader stack.

On 1 November 2012 06:48, Demian Katz <[hidden email]> wrote:

Thanks for this – very encouraging.

 

Just one question: does your configuration rely on none of the modules defining autoloading rules for the root \Soflomo namespace?  Or is there a way to convince the autoloader to load \Soflomo\Log from one place, \Soflomo\Mail from another, and fail back to a third location for remaining \Soflomo classes?  I tried to figure this out a few months ago and failed, but maybe I wasn’t trying hard enough.

 

- Demian

 

The only things you must keep in mind are:

  1. If you link the module to your ./module/ dir, make sure you have the module in ./module/Soflomo/Log and not ./module/<name it here> because the autoloader can't find it. We use composer, so that doesn't bother us.
  2. The autoloader config in the module class can't be the simple __NAMESPACE__ => __DIR__ . '/src/' . __NAMESPACE__, if you (like we have) put a /src/Log/ in your module and not a /src/Soflomo/Log/
  3. As point #2, same holds for the composer.json.

As you notice, they are not real big issues, just some tiny parts which deviate from a standard configuration.


Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

Jurian Sluiman
In reply to this post by demiankatz
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

EvanDotPro
In reply to this post by demiankatz
On Thu, Nov 1, 2012 at 6:45 AM, Demian Katz <[hidden email]> wrote:
>
> Thanks (to you, and everyone else who replied) for the feedback – this is very helpful.
>
>
>
> Regarding my “theme system,” this is a mechanism for creating multiple named sets of templates (that can optionally inherit from one another) to allow multiple themes within an application.  It also has features like automatic switching based on mobile device detection.  I don’t think these things are native framework features, though if I’m reinventing the wheel, please point me in the right direction.

Well, the multiple / inherited templates should work out of the box
with our modules/view layer. Mobile device detection is not native in
ZF2,  you're correct about that.

It sounds like we've answered your original performance concerns, but
also, keep in mind with module loading, if you happened to get into
the hundreds or thousands of modules, you can always generate a
classmap in production that identifies where all the Module classes
are, which will entirely bypass the ModuleAutoloader, and additionally
you can enable config caching which bypasses the ConfigListener. Those
are the only two measurable parts of module loading from a performance
perspective, really, but still nothing you'll notice until you're
loading a massive number of modules. Also, don't forget about
EdpSuperluminal which will also result in the ModuleAutoloader being
bypassed as well.

-Evan
Reply | Threaded
Open this post in threaded view
|

Re: Design/performance question: many modules in a ZF2 application?

Artur Bodera
On Thu, Nov 1, 2012 at 6:49 PM, Evan Coury <[hidden email]> wrote:
 Also, don't forget about
EdpSuperluminal which will also result in the ModuleAutoloader being
bypassed as well.

Well, autoloading can be easily optimized ... but ...

I'd rather point to Module::init() as the bottleneck for having a large number of  (enabled) modules in an app. This also includes all other *aware functionality (and related Module:: methods) which will be each get triggered X times the number of modules, which can multiply quickly and cause a waterfall of function calls, loops etc (which are expensive in PHP).


-- 
      __
     /.)\   +48 695 600 936
     \(./   [hidden email]