PHPUnit taking forever?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

PHPUnit taking forever?

Brian Caspe
Hi,

I've just set up PHPUnit on my project ala http://www.zendcasts.com/unit-testing-with-the-zend-framework-with-zend_test-and-phpunit/2009/06/. Things seem to be working in that it will run tests, but the run time is like 49 seconds (10 mb). This seems like it's terribly long to me.. Is there a way to see what is taking so long? I googled for the answer but couldn't find anything. It seems to me that there is some simple thing that I missed in setting up either my application files, bootstrap or the phpunit testcase. Any help would be appreciated!

Brian

--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Hector Virgen
Do your unit tests set up a database connection? That's usually one of the things that can slow down tests.

Also, how many tests are you running? Are certain tests faster than others?

--
Hector


On Thu, May 6, 2010 at 3:59 PM, Brian Caspe <[hidden email]> wrote:
Hi,

I've just set up PHPUnit on my project ala http://www.zendcasts.com/unit-testing-with-the-zend-framework-with-zend_test-and-phpunit/2009/06/. Things seem to be working in that it will run tests, but the run time is like 49 seconds (10 mb). This seems like it's terribly long to me.. Is there a way to see what is taking so long? I googled for the answer but couldn't find anything. It seems to me that there is some simple thing that I missed in setting up either my application files, bootstrap or the phpunit testcase. Any help would be appreciated!

Brian

--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com

--
Hector Virgen
Reply | Threaded
Open this post in threaded view
|

Fwd: [fw-general] PHPUnit taking forever?

Brian Caspe
The database connection is done in application.ini... should I overwrite it for [testing : production]?
resources.db[] = or somethign like that?

I'm just running 2 or 3 tests, so it's something in the way that phpunit runs. I should probably test phpunit with a straightforward test instead of trying to bootstrap the application to make sure that it runs quickly.

Brian


On Fri, May 7, 2010 at 1:13 AM, Hector Virgen <[hidden email]> wrote:
Do your unit tests set up a database connection? That's usually one of the things that can slow down tests.

Also, how many tests are you running? Are certain tests faster than others?

--
Hector



On Thu, May 6, 2010 at 3:59 PM, Brian Caspe <[hidden email]> wrote:
Hi,

I've just set up PHPUnit on my project ala http://www.zendcasts.com/unit-testing-with-the-zend-framework-with-zend_test-and-phpunit/2009/06/. Things seem to be working in that it will run tests, but the run time is like 49 seconds (10 mb). This seems like it's terribly long to me.. Is there a way to see what is taking so long? I googled for the answer but couldn't find anything. It seems to me that there is some simple thing that I missed in setting up either my application files, bootstrap or the phpunit testcase. Any help would be appreciated!

Brian

--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com




--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Hector Virgen
Can you paste your tests? It shouldn't take that long for 3 tests.

--
Hector


On Thu, May 6, 2010 at 4:53 PM, Brian Caspe <[hidden email]> wrote:
The database connection is done in application.ini... should I overwrite it for [testing : production]?
resources.db[] = or somethign like that?

I'm just running 2 or 3 tests, so it's something in the way that phpunit runs. I should probably test phpunit with a straightforward test instead of trying to bootstrap the application to make sure that it runs quickly.

Brian


On Fri, May 7, 2010 at 1:13 AM, Hector Virgen <[hidden email]> wrote:
Do your unit tests set up a database connection? That's usually one of the things that can slow down tests.

Also, how many tests are you running? Are certain tests faster than others?

--
Hector



On Thu, May 6, 2010 at 3:59 PM, Brian Caspe <[hidden email]> wrote:
Hi,

I've just set up PHPUnit on my project ala http://www.zendcasts.com/unit-testing-with-the-zend-framework-with-zend_test-and-phpunit/2009/06/. Things seem to be working in that it will run tests, but the run time is like 49 seconds (10 mb). This seems like it's terribly long to me.. Is there a way to see what is taking so long? I googled for the answer but couldn't find anything. It seems to me that there is some simple thing that I missed in setting up either my application files, bootstrap or the phpunit testcase. Any help would be appreciated!

Brian

--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com




--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com

--
Hector Virgen
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Brian Caspe
sure! I did a "DummyTest" as described here: http://usingzendframework.blogspot.com/2007/12/setting-up-phpunit.html instead of running it through the zend framework configuration (so it's just loading one file instead of actually bootstrapping my whole application) and the run time again was 47 seconds:
PHPUnit 3.4.12 by Sebastian Bergmann.

F

Time: 47 seconds, Memory: 5.50Mb

There was 1 failure:

1) DummyTest::testFail
Your test successfully failed!

C:\projects\playhouse\tests\DummyTest.php:7

←[37;41m←[2KFAILURES!
←[0m←[37;41m←[2KTests: 1, Assertions: 0, Failures: 1.
←[0m←[2K


On Fri, May 7, 2010 at 2:16 AM, Hector Virgen <[hidden email]> wrote:
Can you paste your tests? It shouldn't take that long for 3 tests.

--
Hector



On Thu, May 6, 2010 at 4:53 PM, Brian Caspe <[hidden email]> wrote:
The database connection is done in application.ini... should I overwrite it for [testing : production]?
resources.db[] = or somethign like that?

I'm just running 2 or 3 tests, so it's something in the way that phpunit runs. I should probably test phpunit with a straightforward test instead of trying to bootstrap the application to make sure that it runs quickly.

Brian


On Fri, May 7, 2010 at 1:13 AM, Hector Virgen <[hidden email]> wrote:
Do your unit tests set up a database connection? That's usually one of the things that can slow down tests.

Also, how many tests are you running? Are certain tests faster than others?

--
Hector



On Thu, May 6, 2010 at 3:59 PM, Brian Caspe <[hidden email]> wrote:
Hi,

I've just set up PHPUnit on my project ala http://www.zendcasts.com/unit-testing-with-the-zend-framework-with-zend_test-and-phpunit/2009/06/. Things seem to be working in that it will run tests, but the run time is like 49 seconds (10 mb). This seems like it's terribly long to me.. Is there a way to see what is taking so long? I googled for the answer but couldn't find anything. It seems to me that there is some simple thing that I missed in setting up either my application files, bootstrap or the phpunit testcase. Any help would be appreciated!

Brian

--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com




--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com




--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Brian Caspe
By the way, this is running on Mamp.

On Fri, May 7, 2010 at 9:13 AM, Brian Caspe <[hidden email]> wrote:
sure! I did a "DummyTest" as described here: http://usingzendframework.blogspot.com/2007/12/setting-up-phpunit.html instead of running it through the zend framework configuration (so it's just loading one file instead of actually bootstrapping my whole application) and the run time again was 47 seconds:
PHPUnit 3.4.12 by Sebastian Bergmann.

F

Time: 47 seconds, Memory: 5.50Mb

There was 1 failure:

1) DummyTest::testFail
Your test successfully failed!

C:\projects\playhouse\tests\DummyTest.php:7

←[37;41m←[2KFAILURES!
←[0m←[37;41m←[2KTests: 1, Assertions: 0, Failures: 1.
←[0m←[2K



On Fri, May 7, 2010 at 2:16 AM, Hector Virgen <[hidden email]> wrote:
Can you paste your tests? It shouldn't take that long for 3 tests.

--
Hector



On Thu, May 6, 2010 at 4:53 PM, Brian Caspe <[hidden email]> wrote:
The database connection is done in application.ini... should I overwrite it for [testing : production]?
resources.db[] = or somethign like that?

I'm just running 2 or 3 tests, so it's something in the way that phpunit runs. I should probably test phpunit with a straightforward test instead of trying to bootstrap the application to make sure that it runs quickly.

Brian


On Fri, May 7, 2010 at 1:13 AM, Hector Virgen <[hidden email]> wrote:
Do your unit tests set up a database connection? That's usually one of the things that can slow down tests.

Also, how many tests are you running? Are certain tests faster than others?

--
Hector



On Thu, May 6, 2010 at 3:59 PM, Brian Caspe <[hidden email]> wrote:
Hi,

I've just set up PHPUnit on my project ala http://www.zendcasts.com/unit-testing-with-the-zend-framework-with-zend_test-and-phpunit/2009/06/. Things seem to be working in that it will run tests, but the run time is like 49 seconds (10 mb). This seems like it's terribly long to me.. Is there a way to see what is taking so long? I googled for the answer but couldn't find anything. It seems to me that there is some simple thing that I missed in setting up either my application files, bootstrap or the phpunit testcase. Any help would be appreciated!

Brian

--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com




--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com




--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

dmitrybelyakov
Hi, Brian.

First of all how much tests do you have? Generally 49 seconds is not so long to test a project. There are still some things to consider:

- Test that use the actual database take longer
- Testing controller and output (like DOM Query) takes even longer
- Generating code coverage reports with your tests also encreases run time.

As all these things increase tests execution you may think of minimizing them or alternatively you can group your tests with @group to run only what's needed at the moment.

It's also a good thing to group tests so that you first run:

- quick tests
- tests with a database
- controller/dispatching/output tests


Dmitry.

Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

dmitrybelyakov
In reply to this post by Brian Caspe
Sorry, somehow missed the post where you say you run only 3 tests. So it's indeed quite a long time for 3 tests and it's very strange.

If you run this test how long will it take?

/**
 * @group runMe
 */
class testingSpeedTest extends Zend_Test_PHPUnit_ControllerTestCase
{
    public function testSpeed()
    {
        $this->assertTrue(true);
    }
}

# phpunit --group runMe
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

dmitrybelyakov
In reply to this post by Brian Caspe
Also you can run phpunit with an option to generate JUnit-like xml report.
Than you can see the time each test runs.
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Brian Caspe
Hey Dmitry,

Thanks for the feedback. I even ran a test that was DummyTest (something like just one assertFalse(false) test) without bootstrapping the application at all and it took the same amount of time... I'll try to run phpunit with the xml report on speed and post the results!

Thanks!

brian

On Sat, May 8, 2010 at 7:48 PM, dmitrybelyakov <[hidden email]> wrote:

Also you can run phpunit with an option to generate JUnit-like xml report.
Than you can see the time each test runs.
--
View this message in context: http://zend-framework-community.634137.n4.nabble.com/PHPUnit-taking-forever-tp2133459p2136327.html
Sent from the Zend Framework mailing list archive at Nabble.com.



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Brian Caspe
Riiight. Ok, so the dummyTest is running fine. I didn't realize that the phpunit.xml file would be loaded whenever I call phpunit unless there is the --no-configuration flag.

So even if there aren't very many tests, then it will load my entire application. Which could be taking 40ish seconds to run through (without any tests to speak of). Does that sound plausible?

Brian



On Sun, May 9, 2010 at 12:07 AM, Brian Caspe <[hidden email]> wrote:
Hey Dmitry,

Thanks for the feedback. I even ran a test that was DummyTest (something like just one assertFalse(false) test) without bootstrapping the application at all and it took the same amount of time... I'll try to run phpunit with the xml report on speed and post the results!

Thanks!

brian


On Sat, May 8, 2010 at 7:48 PM, dmitrybelyakov <[hidden email]> wrote:

Also you can run phpunit with an option to generate JUnit-like xml report.
Than you can see the time each test runs.
--
View this message in context: http://zend-framework-community.634137.n4.nabble.com/PHPUnit-taking-forever-tp2133459p2136327.html
Sent from the Zend Framework mailing list archive at Nabble.com.



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

dmitrybelyakov
Well, actually no.

I think you should still investigate the reason for that. Bootstraping your application is required to do testing of ZendFramework application but there's also phpunit itself that needs to be instantiated to run tests.

So what you can do is create a dummy test that extends PHPUnit_Framework_TestCase and try to run it apart from ZendFramework. This way you can figure out whether it's your application or phpunit instantiation that takes so long.

Here are some actual figures from my experience:

Running a single test that involves bootstraping application and generating code coverage reports takes about 1-2 seconds. Testing the whole project (some 600 tests) without code coverage takes about 1 minute. But again i'm on a fast machine.

Dmitry.
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Fudgz
Hi there,

I'm having exactly the same problem with a XAMPP on windows.

I couldn't find out why the tests are taking so long time, however I've got some php trace information:
http://www.mediafire.com/?ynhag1ykmnh

If someone can help us on this, would be much appreciated
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Fudgz
Ok I finally found out why this was taking a lot of time.

The problems seem to be inside the PHPUnit engine; my phpunit.xml file looked like this :

<filter>
    <whitelist>
      <directory suffix=".php">../application</directory>
      <directory suffix=".php">../library</directory>
      <exclude>
        <directory suffix=".php">../library/Zend</directory>
        <directory suffix=".php">../library/Doctrine</directory>
        <directory suffix=".phtml">../application</directory>
        <file>../application/Bootstrap.php</file>
        <file>../application/scripts/doctrine.php</file>
      </exclude>
    </whitelist>
  </filter>

When you specify a path to the whitelist / blacklist, PHPunit WILL "parse" everything in it, recursivly, but will not include them in the final result if they have been "excluded" (or blacklisted).
This also includes every .svn folders and content (which was also my case)
So with this xml bit, every single bit of my libs (including doctrine and zend) would have been parsed by phpunit.
By changing this xml bit into the following one, my tests now run in 3 seconds (instead of ~50)

<filter>
    <whitelist>
      <directory suffix=".php">../application</directory>
      <directory suffix=".php">../library/lib1_to_test</directory>
      <directory suffix=".php">../library/lib2_to_test</directory>
      <exclude>
        <directory suffix=".phtml">../application</directory>
        <file>../application/Bootstrap.php</file>
        <file>../application/scripts/doctrine.php</file>
      </exclude>
    </whitelist>
  </filter>

I hope this helped you.
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Brian Caspe
Interesting. It seems like that should be the right track for me as well, but I'm not sure exactly how to change my xml to include only the most necessary things: here's the current state:
<filter>
        <whitelist>
            <directory suffix=".php">../application/</directory>
            <exclude>
                <directory suffix=".phtml">../application/</directory>
                <file>../application/Bootstrap.php</file>
                <file>../application/controllers/ErrorController.php</file>
            </exclude>
        </whitelist>
    </filter>

That shouldn't be getting any svn files in it, should it?

On Tue, May 25, 2010 at 6:43 PM, Fudgz <[hidden email]> wrote:

Ok I finally found out why this was taking a lot of time.

The problems seem to be inside the PHPUnit engine; my phpunit.xml file
looked like this :

<filter>
   <whitelist>
     <directory suffix=".php">../application</directory>
     <directory suffix=".php">../library</directory>
     <exclude>
       <directory suffix=".php">../library/Zend</directory>
       <directory suffix=".php">../library/Doctrine</directory>
       <directory suffix=".phtml">../application</directory>
       <file>../application/Bootstrap.php</file>
       <file>../application/scripts/doctrine.php</file>
     </exclude>
   </whitelist>
 </filter>

When you specify a path to the whitelist / blacklist, PHPunit WILL "parse"
everything in it, recursivly, but will not include them in the final result
if they have been "excluded" (or blacklisted).
This also includes every .svn folders and content (which was also my case)
So with this xml bit, every single bit of my libs (including doctrine and
zend) would have been parsed by phpunit.
By changing this xml bit into the following one, my tests now run in 3
seconds (instead of ~50)

<filter>
   <whitelist>
     <directory suffix=".php">../application</directory>
     <directory suffix=".php">../library/lib1_to_test</directory>
     <directory suffix=".php">../library/lib2_to_test</directory>
     <exclude>
       <directory suffix=".phtml">../application</directory>
       <file>../application/Bootstrap.php</file>
       <file>../application/scripts/doctrine.php</file>
     </exclude>
   </whitelist>
 </filter>

I hope this helped you.
--
View this message in context: http://zend-framework-community.634137.n4.nabble.com/PHPUnit-taking-forever-tp2133459p2230316.html
Sent from the Zend Framework mailing list archive at Nabble.com.



--
+420 608 577 012

Prague Playhouse
www.pragueplayhouse.com
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Fudgz
It depends of your project's structure I guess ...

Well I am not an expert, but as my tests on this behavior ran, when you put

<whitelist>
   <directory suffix=".php">../application/</directory>

every single file in the application folder will be parsed, the .svn files, the .random_extension files, etc, even if you've excluded them in a <exclude> afterwards

In fact, you shouldn't even need to exclude
<directory suffix=".phtml">../application/</directory>
since you asked to allow only .php files on your whitelist

But are you saying your unittesting is slow only with these filters ? You don't seem to include any lib (unless your libs are in your application folder)

But basically, I recommand you to whitelist only folders you really want to test
E.g.


<whitelist>
   <directory suffix=".php">../application/models</directory>
   <directory suffix=".php">../application/modules/frontend</directory>
   <directory suffix=".php">../application/modules/backend</directory>
   etc.

Then if you have a "../application/my_bullshit_and_stuff" directory, it won't even be parsed by phpunit
Reply | Threaded
Open this post in threaded view
|

Re: PHPUnit taking forever?

Julien Pauli-3
From my experience, Windows is really slow for PHPUnit.
I think that's NTFS' fault.

I cant run any tests for the framework on Windows, too slow (anyway I dont use that system anymore), lasts hours for the whole framework testing.

Try to run the exact same tests on *nix systems with ext or other FS and give us your results.

Julien

On Sat, May 29, 2010 at 9:53 PM, Fudgz <[hidden email]> wrote:

It depends of your project's structure I guess ...

Well I am not an expert, but as my tests on this behavior ran, when you put

<whitelist>
  <directory suffix=".php">../application/</directory>

every single file in the application folder will be parsed, the .svn files,
the .random_extension files, etc, even if you've excluded them in a
<exclude> afterwards

In fact, you shouldn't even need to exclude
<directory suffix=".phtml">../application/</directory>
since you asked to allow only .php files on your whitelist

But are you saying your unittesting is slow only with these filters ? You
don't seem to include any lib (unless your libs are in your application
folder)

But basically, I recommand you to whitelist only folders you really want to
test
E.g.


<whitelist>
  <directory suffix=".php">../application/models</directory>
  <directory suffix=".php">../application/modules/frontend</directory>
  <directory suffix=".php">../application/modules/backend</directory>
  etc.

Then if you have a "../application/my_bullshit_and_stuff" directory, it
won't even be parsed by phpunit
--
View this message in context: http://zend-framework-community.634137.n4.nabble.com/PHPUnit-taking-forever-tp2133459p2235870.html
Sent from the Zend Framework mailing list archive at Nabble.com.