Tests
- Overview
- Setting Up Unit and Feature Tests
- Running Unit and Feature Tests
- Setting Up Dusk Tests
- Journey Through Dusk Tests
- Running Dusk Tests
- Troubleshooting
Overview
There are some unit tests, some feature tests, and a whole lot of Laravel Dusk tests.
Please do NOT run all the Dusk tests together!
All my tests reside in the Admin app. Yes, there are feature tests in the BasicFrontend app, but these are for the contact form and in some need of work. I chose to keep them there for now.
Setting Up Unit and Feature Tests
MODIFY YOUR "phpunit.xml" CONFIGURATION FILE:
There should already be a "phpunit.xml" file in the admin app's root folder. The only thing that you must modify in this config file is the name of your test database. This test database must be set up before running any tests:
YOU NEED TO SET UP SOMETHING FOR EMAIL:
Emails are sent during a very few of the tests. If email set-up is omitted, these tests will fail.
It is up to you how you want to set up your emails for local development. Here is the Laravel doc.
I am very happy with Mailtrap.io, which has a nice free tier.
RUN "php artisan route:list":
Before you run any test, run on your command line:
php artisan route:list
Seeing all the routes is a good sign! If not, something is awry.
Running Unit and Feature Tests
You can run all the unit and feature tests at once. Run at the command line:
vendor/bin/phpunit
This is what I get (June 2020):
Laravel 7 comes with a new Artisan Test Runner:
php artisan test
Setting Up Dusk Tests
Laravel Dusk runs software tests through the browser. It is fabulous! It's also a pain in the ass.
You need to create three environment files, and modify one existing configuration file, for the Dusk tests:
CREATE THE ".env.dusk.local" ENVIRONMENT FILE:
Copy your ".env" file to ".env.dusk.local".
Change these settings:
CREATE THE ".env.dusk.banallusers" ENVIRONMENT FILE:
Copy the ".env.dusk.local" that you just created to ".env.dusk.banallusers" (pronounced "ban all users"):
CREATE THE ".env.dusk.twofactorauthentication" ENVIRONMENT FILE:
Copy the ".env.dusk.local" that you just created to ".env.dusk.twofactorauthentication":
MODIFY THE ".phpunit.dusk.xml" configuration file:
Please ensure that you specify your test database's name in your "phpunit.dusk.xml":
I MODIFIED "DuskTestCase.php":
"tests/DuskTestCase.php" is a standard base class that comes with Dusk. I modified it:
DUSK TAKES SCREENSHOTS!
Dusk takes a screenshot of the browser at the moment of a test failure. Screenshots are in the "screenshots" folder:
Journey Through Dusk Tests
Dusk goes through the browser step-by-step automatically. The actual page is rendered in the browser (whether it displays during the test or not). Typically, the page is rendered, then an assertion is made that something displays, or is missing, on the page.
Well, the page may take too long to render, and bam! The assertion is made -- and fails!
Hey, there's a real browser going on!
Typical testing scenario:
Typical Dusk snafu: The browser has not finished rendering the login process, but Dusk goes ahead with the assert statement anyways. Well, there is no "Welcome Bob!" message yet, so the assertion fails.
There is a wonderful Dusk method called "pause" that makes the Dusk test do nothing. Dusk pauses so the browser can finish doing what it's doing. Hey Dusk, slow down, you move too fast!
Just one little problem with "pause": you did not pause long enough!
Hey, it's a real browser! Its speed depends on the computer. RAM, CPU, phases of the moon.
In my ongoing efforts to figure out the infamous Dusk question: "how long to pause", I created a method in my custom Dusk base test called (wait for it)... "pause":
As you have already discerned, I failed miserably at understanding the magic number to pause as the "base pause". So, heck, let's just conjure up yet another environment variable so I can get a number high enough for my tests to pass given the particular mood of my computer:
I spent an inordinate amount of time on this aspect of Dusk, frequently on the wrong side of midnight. You can see my anguish in the Dusk tests. Here's an example:
You know that suite of Dusk tests that took 10 minutes to run cleanly and successfully? Well, two weeks ago one of those tests failed even though the test is fine. I ended up elongating a pause. Well, last week that same test failed again. Elongating the pause yet again did the trick!
So, how many times have I run my suite of Dusk tests? A lot. Well, more than merely a lot. Lots!! During a run, if an individual test produced a false negative, I would lengthen the pause(s). So over time, from running my Dusk tests over and over and over, I ended up tuning my Dusk tests to run at the absolute slowest it could run to accommodate the slowest browser rendering my computer could muster at its sleepiest.
If you are feeling optimstic, set the environment variable to 0 (zero) and cross your fingers. If all goes well, you scored major time savings. I set the env var to 500 as my own baseline. The highest I've set this var is 2000.
The entire Dusk suite will take hours.
Running Dusk Tests
I myself do not run my Dusk tests in one fell swoop:
Typically, I use these groupings:
php artisan dusk --group authentication
php artisan dusk --group novablogtables
php artisan dusk --group NovaInstalleddomainsjwtkeystable
php artisan dusk --group NovaInstalleddomainstable
php artisan dusk --group novaLogin
php artisan dusk --group novalookuptables
php artisan dusk --group Client
php artisan dusk --group novaPersonbydomain
The following group of Dusk tests runs out of memory frequently on me:
php artisan dusk --group novaprofiletables
If you encounter the out of memory error, these are the subgroups that comprise the "novaprofiletables" group:
php artisan dusk --group novaaddress
php artisan dusk --group novacompany
php artisan dusk --group novaemail
php artisan dusk --group novaperson
php artisan dusk --group novasocial
php artisan dusk --group novatelephone
php artisan dusk --group novaswebsite
php artisan dusk --group novaprofiletablesPolicies --> which is comprised of the following:
php artisan dusk --group novaprofiletablesPoliciesAddresses
php artisan dusk --group novaprofiletablesPoliciesCompanies
php artisan dusk --group novaprofiletablesPoliciesEmails
php artisan dusk --group novaprofiletablesPoliciesPersons
php artisan dusk --group novaprofiletablesPoliciesSocials
php artisan dusk --group novaprofiletablesPoliciesTelephones
php artisan dusk --group novaprofiletablesPoliciesWebsites
RUNNING THE BAN ALL USERS DUSK TEST
For whatever reason, using custom environment files in the Dusk command line does not work for me. So I created artisan commands to set up the environment variables specifically for this Dusk test:
php artisan lslibrarybackend:dusktestusebanallusersenv
php artisan dusk --group novaPersonbydomainBanuserEmergencybanIsenabled
php artisan lslibrarybackend:dusktestuseregularenv
RUNNING THE TWO FACTOR AUTHENTICATION DUSK TESTS
For whatever reason, using custom environment files in the Dusk command line does not work for me. So I created artisan commands to set up the environment variables specifically for these Dusk tests:
php artisan lslibrarybackend:dusktestusetwofactorauthenv
php artisan dusk --group authenticationTwofactorauthentication
php artisan lslibrarybackend:dusktestuseregularenv
Troubleshooting
All my Dusk tests get "Chrome version must be between __ and __", or "This version of ChromeDriver only supports Chrome version __"
If you get an error that your chrome version is out of date, try this artisan command:
php artisan dusk:chrome-driver
You may have to update your version of Chrome as well.
I get the allowed memory size of nnn bytes exhausted error.
I get this error when I run a suite of Dusk tests. The more Dusk tests I run at the same time, the more likely I will encounter this error. Dusk is launching a new browser over and over again, and I suspect memory management is not efficient.
When I run the individual test that got this error, that test invariably passes.
The only way I avoid this error is running each Dusk test individually. Not helpful.
I usually have a bunch of iTerm windows open. I will exit all of them, open a fresh iTerm window, then run the same Dusk suite again. This actually works a lot. But, not always. When it does not work, I will close desktop apps and browsers. Occassionally I will even reboot (yuk).
The wrong database is used.
Do you know that Laravel changes your "APP_ENV" in the .env whilst your tests are running? It changes it to "testing".
And, do you know that Laravel also changes your "DB_DATABASE" in the .env whilst your tests are running? It changes it to the name of the database specified in your phpunit.xml or phpunit.dusk.xml.
If something should interrupt your tests whilst running, your "APP_ENV" may be "testing" permanently, and your "DB_DATABASE" may be "test db" permanently. So double-check both of these environment variables.
Wow, all my tests are failing!
As obvious as it is, double check that you started your local server, including MySQL (or whatever database you are using).