LaSalle Software

Tests

Overview


There are some unit tests, some feature tests, and a whole lot of Laravel Dusk tests.

Please do NOT run all the Dusk tests together!

All my tests reside in the Admin app. Yes, there are feature tests in the BasicFrontend app, but these are for the contact form and in some need of work. I chose to keep them there for now.

Unit and Feature Tests


Please make sure that your phpunit.xml is set up ok.

Please especially make sure that "DB_DATABASE" is set to the correct database. Make sure that this database exists before running your tests!


Now, before you run any test, run on your command line: 

php artisan route:list

Seeing all the routes is a good sign! If not, something is awry.

You can run all the unit and feature tests at once. Run at the command line: 
vendor/bin/phpunit

This is what I get (April 2020):

Laravel 7 comes with a new Artisan Test Runner: 
php artisan test

Dusk Tests


Laravel Dusk runs software tests through the browser. It is fabulous! It's also a pain in the ass.

You need to set up Dusk first. Please refer to Dusk's documentation. Here are pointers about the ".env.dusk.local" and "phpunit.dusk.xml" files:






Please ensure that you have a "phpunit.dusk.xml" and that the settings are ok. The database must be set up before running Dusk.

You should have "DuskTestCase.php" in your tests folder. All Dusk tests extend this file. Actually, just like I do with controllers, I have my own custom base class that extends DuskTestCase:


Dusk takes a screenshot of the browser at the moment of a test failure. Screenshots are in the "screenshots" folder:


Dusk goes through the browser step-by-step automatically. The actual page is rendered in the browser (whether it displays during the test or not). Typically, the page is rendered, then an assertion is made that something displays, or is missing, on the page.

Well, the page may take too long to render, and bam! The assertion is made -- and fails!

Hey, there's a real browser going on!

Typical testing scenario:
  • Dusk says, "launch the browser", and a browser is launched (you may not see this browser during the test).
  • Then Dusk says "go to the login page", and the browser displays the login page (yes, whether you see it or not).
  • Then, Dusk says "type 'Bob' in the username box", "type 'secret' in th e password box", and then "click the submit button".
  • Then the browser waits for the login to process, and then up comes the "Welcome Bob!" message.
  • The assertion is "see 'Welcome Bob!'".
  • TRUE! Test passes.


    • Typical Dusk snafu: The browser has not finished rendering the login process, but Dusk goes ahead with the assert statement anyways. Well, there is no "Welcome Bob!" message yet, so the assertion fails.

    There is a wonderful Dusk method called "pause" that makes the Dusk test do nothing. Dusk pauses so the browser can finish doing what it's doing. Hey Dusk, slow down, you move too fast!


    Just one little problem with "pause": you did not pause long enough!

    Hey, it's a real browser! Its speed depends on the computer. RAM, CPU, phases of the moon.

    In my ongoing efforts to figure out the infamous Dusk question: "how long to pause", I created a method in my custom Dusk base test called (wait for it)... "pause":


    As you have already discerned, I failed miserably at understanding the magic number to pause as the "base pause". So, heck, let's just conjure up yet another environment variable so I can get a number high enough for my tests to pass given the particular mood of my computer:


    I spent an inordinate amount of time on this aspect of Dusk, frequently on the wrong side of midnight. You can see my anguish in the Dusk tests. Here's an example:


    You know that suite of Dusk tests that took 10 minutes to run cleanly and successfully? Well, two weeks ago one of those tests failed even though the test is fine. I ended up elongating a pause. Well, last week that same test failed again. Elongating the pause yet again did the trick!

    So, how many times have I run my suite of Dusk tests? A lot. Well, more than merely a lot. Lots!! During a run, if an individual test produced a false negative, I would lengthen the pause(s). So over time, from running my Dusk tests over and over and over, I ended up tuning my Dusk tests to run at the absolute slowest it could run to accommodate the slowest browser rendering my computer could muster at its sleepiest.

    If you are feeling optimstic, set the environment variable to 0 (zero) and cross your fingers. If all goes well, you scored major time savings. I set the env var to 2000 as my own baseline.

    The entire Dusk suite will take hours. If you told me it took you three hours, I'd believe you and want to know if your running 3600MHz RAM with a Ryzen 7 CPU.

    I myself do not run my Dusk tests in one fell swoop:


    Typically, I use these groupings: 

    php artisan dusk --group authentication

    php artisan dusk --group novablogtables

    php artisan dusk --group NovaInstalleddomainsjwtkeystable

    php artisan dusk --group NovaInstalleddomainstable

    php artisan dusk --group novaLogin

    php artisan dusk --group novalookuptables

    php artisan dusk --group novaPersonbydomain

    The following group of Dusk tests runs out of memory frequently on me: 
    php artisan dusk --group novaprofiletables

    If you encounter the out of memory error, these are the subgroups that comprise the "novaprofiletables" group: 
    php artisan dusk --group novaaddress
  php artisan dusk --group novacompany
  php artisan dusk --group novaemail
  php artisan dusk --group novaperson
  php artisan dusk --group novasocial
  php artisan dusk --group novatelephone
  php artisan dusk --group novaswebsite
  
  php artisan dusk --group novaprofiletablesPolicies --> which is comprised of the following:
     php artisan dusk --group novaprofiletablesPoliciesAddresses
     php artisan dusk --group novaprofiletablesPoliciesCompanies
     php artisan dusk --group novaprofiletablesPoliciesEmails
     php artisan dusk --group novaprofiletablesPoliciesPersons
     php artisan dusk --group novaprofiletablesPoliciesSocials
     php artisan dusk --group novaprofiletablesPoliciesTelephones
     php artisan dusk --group novaprofiletablesPoliciesWebsites


    Ban All Users Dusk Test


    To ban all users from logging into the admin back-end, the env var "LASALLE_EMERGENCY_BAN_ALL_USERS_FROM_ADMIN_APP_LOGIN" is set to "true".

    The Dusk test to test for the "true" condition needs its own environment file. This file is ".env.dusk.emergencybanuser", and the command to run this Dusk test is:

    php artisan dusk --group novaPersonbydomainBanuserEmergencybanIsenabled --env=dusk.emergencybanuser

    Troubleshooting


    All my Dusk tests get "Chrome version must be between __ and __", or "This version of ChromeDriver only supports Chrome version __"

    If you get an error that your chrome version is out of date, try this artisan command:

    php artisan dusk:chrome-driver

    You may have to update your version of Chrome as well.


    I get the allowed memory size of nnn bytes exhausted error.

    I get this error when I run a suite of Dusk tests. The more Dusk tests I run at the same time, the more likely I will encounter this error. Dusk is launching a new browser over and over again, and I suspect memory management is not efficient.

    When I run the individual test that got this error, that test invariably passes.

    The only way I avoid this error is running each Dusk test individually. Not helpful.

    I usually have a bunch of iTerm windows open. I will exit all of them, open a fresh iTerm window, then run the same Dusk suite again. This actually works a lot. But, not always. When it does not work, I will close desktop apps and browsers. Occassionally I will even reboot (yuk).


    The wrong database is used.

    Do you know that Laravel changes your "APP_ENV" in the .env whilst your tests are running? It changes it to "testing".

    And, do you know that Laravel also changes your "DB_DATABASE" in the .env whilst your tests are running? It changes it to the name of the database specified in your phpunit.xml or phpunit.dusk.xml.

    If something should interrupt your tests whilst running, your "APP_ENV" may be "testing" permanently, and your "DB_DATABASE" may be "test db" permanently. So double-check both of these environment variables.


    Wow, all my tests are failing!

    As obvious as it is, double check that you started your local server, including MySQL (or whatever database you are using).