LaSalle Software

Tests

Overview


There are some unit tests, some feature tests, and a whole lot of Laravel Dusk tests.

Please do NOT run all the Dusk tests together!

All my tests reside in the Admin app. Yes, there are feature tests in the BasicFrontend app, but these are for the contact form and in some need of work. I chose to keep them there for now.

Setting Up Unit and Feature Tests


MODIFY YOUR "phpunit.xml" CONFIGURATION FILE:


There should already be a "phpunit.xml" file in the admin app's root folder. The only thing that you must modify in this config file is the name of your test database. This test database must be set up before running any tests:


YOU NEED TO SET UP SOMETHING FOR EMAIL:


Emails are sent during a very few of the tests. If email set-up is omitted, these tests will fail.

It is up to you how you want to set up your emails for local development. Here is the Laravel doc.

I am very happy with Mailtrap.io, which has a nice free tier.

RUN "php artisan route:list":


Before you run any test, run on your command line: 

php artisan route:list

Seeing all the routes is a good sign! If not, something is awry.

Running Unit and Feature Tests


You can run all the unit and feature tests at once. Run at the command line: 
vendor/bin/phpunit

This is what I get (June 2020):

Laravel 7 comes with a new Artisan Test Runner: 
php artisan test

Setting Up Dusk Tests


Laravel Dusk runs software tests through the browser. It is fabulous! It's also a pain in the ass.

You need to create three environment files, and modify one existing configuration file, for the Dusk tests:
  • .env.dusk.local
  • .env.dusk.banallusers
  • .env.dusk.twofactorauthentication
  • phpunit.dusk.xml

    • CREATE THE ".env.dusk.local" ENVIRONMENT FILE:


    Copy your ".env" file to ".env.dusk.local".

    Change these settings:


    CREATE THE ".env.dusk.banallusers" ENVIRONMENT FILE:


    Copy the ".env.dusk.local" that you just created to ".env.dusk.banallusers" (pronounced "ban all users"):


    CREATE THE ".env.dusk.twofactorauthentication" ENVIRONMENT FILE:


    Copy the ".env.dusk.local" that you just created to ".env.dusk.twofactorauthentication":


    MODIFY THE ".phpunit.dusk.xml" configuration file:


    Please ensure that you specify your test database's name in your "phpunit.dusk.xml":


    I MODIFIED "DuskTestCase.php":


    "tests/DuskTestCase.php" is a standard base class that comes with Dusk. I modified it:


    DUSK TAKES SCREENSHOTS!


    Dusk takes a screenshot of the browser at the moment of a test failure. Screenshots are in the "screenshots" folder:


    Journey Through Dusk Tests


    Dusk goes through the browser step-by-step automatically. The actual page is rendered in the browser (whether it displays during the test or not). Typically, the page is rendered, then an assertion is made that something displays, or is missing, on the page.

    Well, the page may take too long to render, and bam! The assertion is made -- and fails!

    Hey, there's a real browser going on!

    Typical testing scenario:
  • Dusk says, "launch the browser", and a browser is launched (you may not see this browser during the test).
  • Then Dusk says "go to the login page", and the browser displays the login page (yes, whether you see it or not).
  • Then, Dusk says "type 'Bob' in the username box", "type 'secret' in th e password box", and then "click the submit button".
  • Then the browser waits for the login to process, and then up comes the "Welcome Bob!" message.
  • The assertion is "see 'Welcome Bob!'".
  • TRUE! Test passes.


    • Typical Dusk snafu: The browser has not finished rendering the login process, but Dusk goes ahead with the assert statement anyways. Well, there is no "Welcome Bob!" message yet, so the assertion fails.

    There is a wonderful Dusk method called "pause" that makes the Dusk test do nothing. Dusk pauses so the browser can finish doing what it's doing. Hey Dusk, slow down, you move too fast!


    Just one little problem with "pause": you did not pause long enough!

    Hey, it's a real browser! Its speed depends on the computer. RAM, CPU, phases of the moon.

    In my ongoing efforts to figure out the infamous Dusk question: "how long to pause", I created a method in my custom Dusk base test called (wait for it)... "pause":


    As you have already discerned, I failed miserably at understanding the magic number to pause as the "base pause". So, heck, let's just conjure up yet another environment variable so I can get a number high enough for my tests to pass given the particular mood of my computer:


    I spent an inordinate amount of time on this aspect of Dusk, frequently on the wrong side of midnight. You can see my anguish in the Dusk tests. Here's an example:


    You know that suite of Dusk tests that took 10 minutes to run cleanly and successfully? Well, two weeks ago one of those tests failed even though the test is fine. I ended up elongating a pause. Well, last week that same test failed again. Elongating the pause yet again did the trick!

    So, how many times have I run my suite of Dusk tests? A lot. Well, more than merely a lot. Lots!! During a run, if an individual test produced a false negative, I would lengthen the pause(s). So over time, from running my Dusk tests over and over and over, I ended up tuning my Dusk tests to run at the absolute slowest it could run to accommodate the slowest browser rendering my computer could muster at its sleepiest.

    If you are feeling optimstic, set the environment variable to 0 (zero) and cross your fingers. If all goes well, you scored major time savings. I set the env var to 500 as my own baseline. The highest I've set this var is 2000.

    The entire Dusk suite will take hours.

    Running Dusk Tests


    I myself do not run my Dusk tests in one fell swoop:


    Typically, I use these groupings: 

    php artisan dusk --group authentication

    php artisan dusk --group novablogtables

    php artisan dusk --group NovaInstalleddomainsjwtkeystable

    php artisan dusk --group NovaInstalleddomainstable

    php artisan dusk --group novaLogin

    php artisan dusk --group novalookuptables

    php artisan dusk --group Client

    php artisan dusk --group novaPersonbydomain

    The following group of Dusk tests runs out of memory frequently on me: 
    php artisan dusk --group novaprofiletables

    If you encounter the out of memory error, these are the subgroups that comprise the "novaprofiletables" group: 
    php artisan dusk --group novaaddress
  php artisan dusk --group novacompany
  php artisan dusk --group novaemail
  php artisan dusk --group novaperson
  php artisan dusk --group novasocial
  php artisan dusk --group novatelephone
  php artisan dusk --group novaswebsite
  
  php artisan dusk --group novaprofiletablesPolicies --> which is comprised of the following:
     php artisan dusk --group novaprofiletablesPoliciesAddresses
     php artisan dusk --group novaprofiletablesPoliciesCompanies
     php artisan dusk --group novaprofiletablesPoliciesEmails
     php artisan dusk --group novaprofiletablesPoliciesPersons
     php artisan dusk --group novaprofiletablesPoliciesSocials
     php artisan dusk --group novaprofiletablesPoliciesTelephones
     php artisan dusk --group novaprofiletablesPoliciesWebsites


    RUNNING THE BAN ALL USERS DUSK TEST


    For whatever reason, using custom environment files in the Dusk command line does not work for me. So I created artisan commands to set up the environment variables specifically for this Dusk test:
    php artisan lslibrarybackend:dusktestusebanallusersenv
    php artisan dusk --group novaPersonbydomainBanuserEmergencybanIsenabled
    php artisan lslibrarybackend:dusktestuseregularenv


    RUNNING THE TWO FACTOR AUTHENTICATION DUSK TESTS


    For whatever reason, using custom environment files in the Dusk command line does not work for me. So I created artisan commands to set up the environment variables specifically for these Dusk tests:
    php artisan lslibrarybackend:dusktestusetwofactorauthenv
    php artisan dusk --group authenticationTwofactorauthentication
    php artisan lslibrarybackend:dusktestuseregularenv


    Troubleshooting


    All my Dusk tests get "Chrome version must be between __ and __", or "This version of ChromeDriver only supports Chrome version __"

    If you get an error that your chrome version is out of date, try this artisan command:

    php artisan dusk:chrome-driver

    You may have to update your version of Chrome as well.


    I get the allowed memory size of nnn bytes exhausted error.

    I get this error when I run a suite of Dusk tests. The more Dusk tests I run at the same time, the more likely I will encounter this error. Dusk is launching a new browser over and over again, and I suspect memory management is not efficient.

    When I run the individual test that got this error, that test invariably passes.

    The only way I avoid this error is running each Dusk test individually. Not helpful.

    I usually have a bunch of iTerm windows open. I will exit all of them, open a fresh iTerm window, then run the same Dusk suite again. This actually works a lot. But, not always. When it does not work, I will close desktop apps and browsers. Occassionally I will even reboot (yuk).


    The wrong database is used.

    Do you know that Laravel changes your "APP_ENV" in the .env whilst your tests are running? It changes it to "testing".

    And, do you know that Laravel also changes your "DB_DATABASE" in the .env whilst your tests are running? It changes it to the name of the database specified in your phpunit.xml or phpunit.dusk.xml.

    If something should interrupt your tests whilst running, your "APP_ENV" may be "testing" permanently, and your "DB_DATABASE" may be "test db" permanently. So double-check both of these environment variables.


    Wow, all my tests are failing!

    As obvious as it is, double check that you started your local server, including MySQL (or whatever database you are using).