Best practices for managing ZF2 modules and dependencies

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Best practices for managing ZF2 modules and dependencies

ajbdev
I'm developing a ZF2 app that will make heavy use of feature specific modules.

While modules will be as atomic as possible, certain modules will be dependent on other modules existing, and nearly all modules will require at least one core module to exist in order to function.

However, this application is going to be deployed to several different sites, and not all sites will have all modules. Some sites will have modules that other sites don't have. It's important that I am able to easily manage modules per site without a code change and without forking the codebase off for individual sites.

Ideally, I'd love to manage these modules using composer.
It seems like in an ideal world I would have a base composer.json-dist with the basic modules that all site deployments would have, and then that file would be copied and modified as necessary as part of the install/update process. Then each module itself would have it's own composer file that would be used to track it's own dependencies (which would also lend itself nicely to handling third party libraries, too).

Where I start to get lost is the development workflow for the application. Each module will need it's own repository, but I don't want to have to open up one codebase per module - ideally I want to develop in a single codebase that would contain the application and all modules, but still allow me to commit and push individually to each module repo. Is there a good way to do that?

Maybe I'm going at this the completely wrong way and there's a better way to achieve what I'm looking for. If so, I'd love to hear it.  


Reply | Threaded
Open this post in threaded view
|

Re: Best practices for managing ZF2 modules and dependencies

Mischosch

Am 12.02.2015 um 00:37 schrieb ajbdev <[hidden email]>:

> I'm developing a ZF2 app that will make heavy use of feature specific
> modules.
>
> While modules will be as atomic as possible, certain modules will be
> dependent on other modules existing, and nearly all modules will require at
> least one core module to exist in order to function.
>
> However, this application is going to be deployed to several different
> sites, and not all sites will have all modules. Some sites will have modules
> that other sites don't have. It's important that I am able to easily manage
> modules per site without a code change and without forking the codebase off
> for individual sites.
>
> Ideally, I'd love to manage these modules using composer.
> It seems like in an ideal world I would have a base composer.json-dist with
> the basic modules that all site deployments would have, and then that file
> would be copied and modified as necessary as part of the install/update
> process. Then each module itself would have it's own composer file that
> would be used to track it's own dependencies (which would also lend itself
> nicely to handling third party libraries, too).
>
> Where I start to get lost is the development workflow for the application.
> Each module will need it's own repository, but I don't want to have to open
> up one codebase per module - ideally I want to develop in a single codebase
> that would contain the application and all modules, but still allow me to
> commit and push individually to each module repo. Is there a good way to do
> that?
>
> Maybe I'm going at this the completely wrong way and there's a better way to
> achieve what I'm looking for. If so, I'd love to hear it.  

Hey there,

you should look into —prefer-source option of composer.

$ composer update symfony/yaml --prefer-source

By this way you can work inside your project with many repos under the hood. So yes, a single repository each module should be the way to go for your described needs.


Regards

Michael



--
List: [hidden email]
Info: http://framework.zend.com/archives
Unsubscribe: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Best practices for managing ZF2 modules and dependencies

Vincent Caggiari
In reply to this post by ajbdev
Hello,

We tried exactly the same thing (same modules for different clients) but
things got so complicated to manage that we stopped to build websites this
way.
Why it became so complicated :

   - Same code base for different clients doesn't allow you to customize
   them so when a client has a specific use case/need, you can't do it (and
   don't tell me that you will create a new module for this client which will
   override the first one for the parts you need to customize because you will
   have 2 modules doing the same thing and when you will have to do some
   changes, you will not remember in which one to look).
   - You MUST have one repository per module, it's easier to manage and see
   what your developers are doing. If you need to update only one module (i.e
   patch), you can then update your project easily with only the change you
   need (so you don't pull changes made on another module which can introduce
   bugs for a specific client or some merging headache).


At this time, we have one repository per module. Each module for each
client is forked from our "module" core codebase so when a change is made
for a client and can be used by another (generic change), you update your
client module, then make a pull request to the "module" core codebase which
is then approved and incorporated.
Then, when you need to work on another client which use the same module,
you update your client's one by pulling the "module" core codebase, merge
the changes with the customized one, test and voila.

Regards,
CAGGIARI Vincent.
Reply | Threaded
Open this post in threaded view
|

Re: Best practices for managing ZF2 modules and dependencies

ajbdev
Hi everyone,

Thank you all for the fantastic advice. I am now working on implementing a
solution that uses --prefer-source and I'm running into two problems that
Google didn't solve.

I apologize as these are more composer specific than Zend, but it sounds
like several members may have experience with this already.

   - Is there a way to set --prefer-source on a per package basis? I found
   the config stub option for composer.json, but that seems to affect all
   packages.
   - Like the first question, is there a way to set a specific directory
   for the checkout of specific packages? For example, I'd love to clone the
   custom modules into the module directory instead of /vendor. Again I see
   how I can set this globally, but I'm unsure if this can be done on a per
   package basis.

Thanks,

-Andy


On Thu, Feb 12, 2015 at 1:10 AM, Michael Gooden <[hidden email]
> wrote:

> Hi,
>
> You could also take the approach similiar to Zend Framework 2 itself,
> wherein all the code lies in a single repository, with a master
> composer.json that does a "replace self.version" for all your modules. You
> use this during development with --prefer-source, which should make it easy
> to develop the code.
>
> In parallel, you maintain composer.json files within each module path that
> describe the dependencies of that specific modules. For production
> deployment, you use the script that ZF2 uses, to break the main repo into
> individual composer repositories.
>
> I must say, Vincent's approach does seem more structured than this. YMMV.
>
> ​
> Kind Regards,
>
> *Michael Gooden*
> CEO
>
> ​
> ​​*Blue Point Events (Pty) Ltd*
>
> Jutland Crescent, St Georges Park, Port Elizabeth, 6001
>
> PO Box 63941, Greenacres, Port Elizabeth, 6057
>
> M: +27 76 489 1764 | F: +27 86 551 2556
>
> [hidden email]
>
> Skype: michael.bluepointweb​
>
> *NOTICE OF CONFIDENTIALITY*
> The preceding e-mail message (including any attachments) contains
> information that may be confidential, may be protected by the
> attorney-client or other applicable privileges, or may constitute
> non-public information. It is intended to be conveyed only to the
> designated recipient(s) named above. If you are not an intended recipient
> of this message, please notify the sender by replying to this message and
> then delete all copies of it from your computer system. Any use,
> dissemination, distribution, or reproduction of this message by unintended
> recipients is not authorized and may be unlawful.
>
> On 12 February 2015 at 02:48, Vincent Caggiari <[hidden email]>
> wrote:
>
>> Hello,
>>
>> We tried exactly the same thing (same modules for different clients) but
>> things got so complicated to manage that we stopped to build websites this
>> way.
>> Why it became so complicated :
>>
>>    - Same code base for different clients doesn't allow you to customize
>>    them so when a client has a specific use case/need, you can't do it
>> (and
>>    don't tell me that you will create a new module for this client which
>> will
>>    override the first one for the parts you need to customize because you
>> will
>>    have 2 modules doing the same thing and when you will have to do some
>>    changes, you will not remember in which one to look).
>>    - You MUST have one repository per module, it's easier to manage and
>> see
>>    what your developers are doing. If you need to update only one module
>> (i.e
>>    patch), you can then update your project easily with only the change
>> you
>>    need (so you don't pull changes made on another module which can
>> introduce
>>    bugs for a specific client or some merging headache).
>>
>>
>> At this time, we have one repository per module. Each module for each
>> client is forked from our "module" core codebase so when a change is made
>> for a client and can be used by another (generic change), you update your
>> client module, then make a pull request to the "module" core codebase
>> which
>> is then approved and incorporated.
>> Then, when you need to work on another client which use the same module,
>> you update your client's one by pulling the "module" core codebase, merge
>> the changes with the customized one, test and voila.
>>
>> Regards,
>> CAGGIARI Vincent.
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Best practices for managing ZF2 modules and dependencies

tasmaniski
Yes.

This is related to composer, so  actually there is repository type Artifact for local repositories:
https://getcomposer.org/doc/05-repositories.md#artifact

Or, you can put in composer.json path to your git server (or github)  for packages.

Not sure what you are looking for, but check the link above.
Reply | Threaded
Open this post in threaded view
|

Re: Best practices for managing ZF2 modules and dependencies

ajbdev
In reply to this post by ajbdev
After doing some more searching I came upon this:

https://getcomposer.org/doc/faqs/how-do-i-install-a-package-to-a-custom-path-for-my-framework.md

I ended up implementing this and it worked perfectly having my package use
composer/install with the "zend-module" type.

Thanks for the help all!

On Wed, Mar 11, 2015 at 11:09 PM, Andy Baird <[hidden email]> wrote:

> Hi everyone,
>
> Thank you all for the fantastic advice. I am now working on implementing a
> solution that uses --prefer-source and I'm running into two problems that
> Google didn't solve.
>
> I apologize as these are more composer specific than Zend, but it sounds
> like several members may have experience with this already.
>
>    - Is there a way to set --prefer-source on a per package basis? I
>    found the config stub option for composer.json, but that seems to affect
>    all packages.
>    - Like the first question, is there a way to set a specific directory
>    for the checkout of specific packages? For example, I'd love to clone the
>    custom modules into the module directory instead of /vendor. Again I see
>    how I can set this globally, but I'm unsure if this can be done on a per
>    package basis.
>
> Thanks,
>
> -Andy
>
>
> On Thu, Feb 12, 2015 at 1:10 AM, Michael Gooden <
> [hidden email]> wrote:
>
>> Hi,
>>
>> You could also take the approach similiar to Zend Framework 2 itself,
>> wherein all the code lies in a single repository, with a master
>> composer.json that does a "replace self.version" for all your modules. You
>> use this during development with --prefer-source, which should make it easy
>> to develop the code.
>>
>> In parallel, you maintain composer.json files within each module path
>> that describe the dependencies of that specific modules. For production
>> deployment, you use the script that ZF2 uses, to break the main repo into
>> individual composer repositories.
>>
>> I must say, Vincent's approach does seem more structured than this. YMMV.
>>
>> ​
>> Kind Regards,
>>
>> *Michael Gooden*
>> CEO
>>
>> ​
>> ​​*Blue Point Events (Pty) Ltd*
>>
>> Jutland Crescent, St Georges Park, Port Elizabeth, 6001
>>
>> PO Box 63941, Greenacres, Port Elizabeth, 6057
>>
>> M: +27 76 489 1764 | F: +27 86 551 2556
>>
>> [hidden email]
>>
>> Skype: michael.bluepointweb​
>>
>> *NOTICE OF CONFIDENTIALITY*
>> The preceding e-mail message (including any attachments) contains
>> information that may be confidential, may be protected by the
>> attorney-client or other applicable privileges, or may constitute
>> non-public information. It is intended to be conveyed only to the
>> designated recipient(s) named above. If you are not an intended recipient
>> of this message, please notify the sender by replying to this message and
>> then delete all copies of it from your computer system. Any use,
>> dissemination, distribution, or reproduction of this message by unintended
>> recipients is not authorized and may be unlawful.
>>
>> On 12 February 2015 at 02:48, Vincent Caggiari <[hidden email]>
>> wrote:
>>
>>> Hello,
>>>
>>> We tried exactly the same thing (same modules for different clients) but
>>> things got so complicated to manage that we stopped to build websites
>>> this
>>> way.
>>> Why it became so complicated :
>>>
>>>    - Same code base for different clients doesn't allow you to customize
>>>    them so when a client has a specific use case/need, you can't do it
>>> (and
>>>    don't tell me that you will create a new module for this client which
>>> will
>>>    override the first one for the parts you need to customize because
>>> you will
>>>    have 2 modules doing the same thing and when you will have to do some
>>>    changes, you will not remember in which one to look).
>>>    - You MUST have one repository per module, it's easier to manage and
>>> see
>>>    what your developers are doing. If you need to update only one module
>>> (i.e
>>>    patch), you can then update your project easily with only the change
>>> you
>>>    need (so you don't pull changes made on another module which can
>>> introduce
>>>    bugs for a specific client or some merging headache).
>>>
>>>
>>> At this time, we have one repository per module. Each module for each
>>> client is forked from our "module" core codebase so when a change is made
>>> for a client and can be used by another (generic change), you update your
>>> client module, then make a pull request to the "module" core codebase
>>> which
>>> is then approved and incorporated.
>>> Then, when you need to work on another client which use the same module,
>>> you update your client's one by pulling the "module" core codebase, merge
>>> the changes with the customized one, test and voila.
>>>
>>> Regards,
>>> CAGGIARI Vincent.
>>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Best practices for managing ZF2 modules and dependencies

latterd
Take a look at https://github.com/composer/satis I use it at my work to
manage self contained modules, etc...

On Fri, 13 Mar 2015 22:24 Andy Baird <[hidden email]> wrote:

> After doing some more searching I came upon this:
>
> https://getcomposer.org/doc/faqs/how-do-i-install-a-
> package-to-a-custom-path-for-my-framework.md
>
> I ended up implementing this and it worked perfectly having my package use
> composer/install with the "zend-module" type.
>
> Thanks for the help all!
>
> On Wed, Mar 11, 2015 at 11:09 PM, Andy Baird <[hidden email]> wrote:
>
> > Hi everyone,
> >
> > Thank you all for the fantastic advice. I am now working on implementing
> a
> > solution that uses --prefer-source and I'm running into two problems that
> > Google didn't solve.
> >
> > I apologize as these are more composer specific than Zend, but it sounds
> > like several members may have experience with this already.
> >
> >    - Is there a way to set --prefer-source on a per package basis? I
> >    found the config stub option for composer.json, but that seems to
> affect
> >    all packages.
> >    - Like the first question, is there a way to set a specific directory
> >    for the checkout of specific packages? For example, I'd love to clone
> the
> >    custom modules into the module directory instead of /vendor. Again I
> see
> >    how I can set this globally, but I'm unsure if this can be done on a
> per
> >    package basis.
> >
> > Thanks,
> >
> > -Andy
> >
> >
> > On Thu, Feb 12, 2015 at 1:10 AM, Michael Gooden <
> > [hidden email]> wrote:
> >
> >> Hi,
> >>
> >> You could also take the approach similiar to Zend Framework 2 itself,
> >> wherein all the code lies in a single repository, with a master
> >> composer.json that does a "replace self.version" for all your modules.
> You
> >> use this during development with --prefer-source, which should make it
> easy
> >> to develop the code.
> >>
> >> In parallel, you maintain composer.json files within each module path
> >> that describe the dependencies of that specific modules. For production
> >> deployment, you use the script that ZF2 uses, to break the main repo
> into
> >> individual composer repositories.
> >>
> >> I must say, Vincent's approach does seem more structured than this.
> YMMV.
> >>
> >> ​
> >> Kind Regards,
> >>
> >> *Michael Gooden*
> >> CEO
> >>
> >> ​
> >> ​​*Blue Point Events (Pty) Ltd*
> >>
> >> Jutland Crescent, St Georges Park, Port Elizabeth, 6001
> >>
> >> PO Box 63941, Greenacres, Port Elizabeth, 6057
> >>
> >> M: +27 76 489 1764 | F: +27 86 551 2556
> >>
> >> [hidden email]
> >>
> >> Skype: michael.bluepointweb​
> >>
> >> *NOTICE OF CONFIDENTIALITY*
> >> The preceding e-mail message (including any attachments) contains
> >> information that may be confidential, may be protected by the
> >> attorney-client or other applicable privileges, or may constitute
> >> non-public information. It is intended to be conveyed only to the
> >> designated recipient(s) named above. If you are not an intended
> recipient
> >> of this message, please notify the sender by replying to this message
> and
> >> then delete all copies of it from your computer system. Any use,
> >> dissemination, distribution, or reproduction of this message by
> unintended
> >> recipients is not authorized and may be unlawful.
> >>
> >> On 12 February 2015 at 02:48, Vincent Caggiari <
> [hidden email]>
> >> wrote:
> >>
> >>> Hello,
> >>>
> >>> We tried exactly the same thing (same modules for different clients)
> but
> >>> things got so complicated to manage that we stopped to build websites
> >>> this
> >>> way.
> >>> Why it became so complicated :
> >>>
> >>>    - Same code base for different clients doesn't allow you to
> customize
> >>>    them so when a client has a specific use case/need, you can't do it
> >>> (and
> >>>    don't tell me that you will create a new module for this client
> which
> >>> will
> >>>    override the first one for the parts you need to customize because
> >>> you will
> >>>    have 2 modules doing the same thing and when you will have to do
> some
> >>>    changes, you will not remember in which one to look).
> >>>    - You MUST have one repository per module, it's easier to manage and
> >>> see
> >>>    what your developers are doing. If you need to update only one
> module
> >>> (i.e
> >>>    patch), you can then update your project easily with only the change
> >>> you
> >>>    need (so you don't pull changes made on another module which can
> >>> introduce
> >>>    bugs for a specific client or some merging headache).
> >>>
> >>>
> >>> At this time, we have one repository per module. Each module for each
> >>> client is forked from our "module" core codebase so when a change is
> made
> >>> for a client and can be used by another (generic change), you update
> your
> >>> client module, then make a pull request to the "module" core codebase
> >>> which
> >>> is then approved and incorporated.
> >>> Then, when you need to work on another client which use the same
> module,
> >>> you update your client's one by pulling the "module" core codebase,
> merge
> >>> the changes with the customized one, test and voila.
> >>>
> >>> Regards,
> >>> CAGGIARI Vincent.
> >>>
> >>
> >>
> >
>