Testing

Polylith encourages a test-centric approach when working with code. New brick tests are easy to write, and mocking can be avoided in most cases as we have access to all components from the projects they live in.
Let's go back to our example.
Nothing is marked to be tested at the moment, but if we change the core namespace in the user component by adding an extra !, that should do the trick:
(ns se.example.user.core)
​
(defn hello [name]
  (str "Hello " name "!!"))
We can verify that the tool recognises the change by running the diff command, which will give us this output:
components/user/src/se/example/user/core.clj
...and if we run the info command again:
...the user component is now marked with an asterisk, *. If we look carefully we will also notice that the status flags stx under the cl column now has an x in its last position. As we already know, this means that the tests for user and cli will be executed from the command-line project if we execute the test command.
But why is cli marked to be tested? The reason is that even though cli itself hasn't changed, it depends on something that has, namely the user component.
The columns under the development project are all marked as st-. The reason the development project is not marked to be tested is that the development project's tests are not included by default.
But before we run the test command, we should first add a test by editing the interface-test namespace in the user component:
(ns se.example.user.interface-test
  (:require [clojure.test :refer :all]
            [se.example.user.interface :as user]))
​
(deftest hello--when-called-with-a-name--then-return-hello-phrase
  (is (= "Hello Lisa!"
         (user/hello "Lisa"))))
Now we can run the test from the IDE:
Make sure the namespace is loaded, e.g. via the menu (or keyboard shortcuts) Tools > REPL > Load File in REPL
Run the test, e.g:
Run all tests in the current namespace: Tools > REPL > Run Tests in Current NS in REPL
Or, place the cursor under the test and run: Tools > REPL > Run Test under carret in REPL
Oops, the test failed!
And if we run the test command:
poly test
...it fails here too:
Projects to run tests from: command-line
​
Running tests for the command-line project using test runner: Polylith built-in clojure.test runner...
Running tests from the command-line project, including 2 bricks: user, cli
​
Testing se.example.cli.core-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing se.example.user.interface-test
​
FAIL in (hello--when-called-with-a-name--then-return-hello-phrase) (interface_test.clj:6)
expected: (= "Hello Lisa!" (user/hello "Lisa"))
  actual: (not (= "Hello Lisa!" "Hello Lisa!!"))
​
Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
​
Test results: 0 passes, 1 failures, 0 errors.
Remember that we added an extra ! so now we need to update the corresponding test accordingly:
(ns se.example.user.interface-test
  (:require [clojure.test :refer :all]
            [se.example.user.interface :as user]))
​
(deftest hello--when-called-with-a-name--then-return-hello-phrase
  (is (= "Hello Lisa!!"
         (user/hello "Lisa"))))
If we run the test again from the REPL, it will now turn to green:
...and the test command will pass too:
Projects to run tests from: command-line
​
Running tests for the command-line project using test runner: Polylith built-in clojure.test runner...
Running tests from the command-line project, including 2 bricks: user, cli
​
Testing se.example.cli.core-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing se.example.user.interface-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Execution time: x seconds
We have already mentioned that the brick tests will not be executed from the development project when we run the test command. But there is a way to do that, and that is to pass in :dev.
Let's try it out with the info command first:
poly info :dev
Now both the development and the command-line project is marked for test execution. 
Note: In version 0.2.13-alpha and earlier, project:dev was an alias for :dev, which meant that it behaved the same as passing in project:dev and as a result, it only included development in the test execution.
From version 0.2.14-alpha and later, this has changed so that :dev means "also include development" without excluding any other projects. Another difference is that poly test :all project:p1 in 0.2.13-alpha ran all tests for all projects (because we passed in :all) but from version 0.2.14-alpha that statement means "run all tests for project p1".
Filter on projects
We can narrow the number of projects we want to test by passing in e.g. project:dev or project:cl:dev (both project alias and name can be used). If development is given, it will automatically be included as if :dev was passed in:
poly info project:dev
Here we include both projects:
poly info project:cl:dev
Filter on bricks
It's not just possible to filter which projects to run our tests from, but also which bricks to include.
Right now our workspace looks like this:
poly info
Both bricks in the cl project are marked to be tested.
If we select the cli brick:
poly info brick:cli
...now only that brick is marked to be tested.
Let's pretend that no bricks were marked to be tested:
If we run the same command again:
poly info brick:cli
...we get the same result, and that's because the brick:cli parameter is just a filter that is applied after the other status calculations have been performed.
If we want to force the cli tests to be executed, we need to pass in :all-bricks (or :all if we also want to execute the project tests):
poly info brick:cli :all-bricks
Finally, the cli brick is now marked to be tested!
It's also possible to give more than one brick, e.g. brick:cli:user. Another trick we can do is to exclude all bricks with brick:- which can be useful in combination with :project or :all to execute only the project tests.
Project tests
Before we execute any tests, let's add a project test for the command-line project.
Begin by adding a test directory for the command-line project:
example
├── projects
│   └── command-line
│       └── test
Then add the "test" path to projects/command-line/deps.edn:
 :aliases {:test {:extra-paths ["test"]
                  :extra-deps  {}}
...and to ./deps.edn:
            :test {:extra-paths ["components/user/test"
                                 "bases/cli/test"
                                 "projects/command-line/test"]}
Now add the project.command-line.dummy-test namespace to the command-line project:
example
├── projects
│   └── command-line
│       └── test
│           └── project
│               └──command_line
│                  └──dummy_test.clj
(ns project.command-line.dummy-test
  (:require [clojure.test :refer :all]))
​
(deftest dummy-test
  (is (= 1 1)))
We could have chosen another top namespace, e.g., se.example.project.command-line, as long as we don't have any brick with the name project. But because we don't want to get into any name conflicts with bricks and also because each project is executed in isolation, the choice of namespace is less important and here we choose the project.command-line top namespace to keep it simple.
Normally, we are forced to put our tests in the same namespace as the code we want to test, to get proper access, but in Polylith the encapsulation is guaranteed by the poly tool and all code can therefore be declared public, which allows us to put the test code wherever we want.
If we execute the info command:
...the command-line is marked as changed and flagged as -t- telling us that it now has a test directory. The -t- says that it has been added to the development project. The reason it's not tagged as -tx is that project tests are not marked to be executed without explicitly telling them to, by passing in :project.
poly info :project
Now the command-line project is also marked to be tested. Let's verify that by running the tests:
poly test :project
Projects to run tests from: command-line
​
Running tests for the command-line project using test runner: Polylith built-in clojure.test runner...
Running tests from the command-line project, including 2 bricks and 1 project: user, cli, command-line
​
Testing se.example.cli.core-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing se.example.user.interface-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing project.command-line.dummy_test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Execution time: 1 seconds
They passed!
Test approaches
As you have just seen, with Polylith we can add tests at two different levels: brick and project.
The project tests should be used for our slow tests, e.g. tests that take more than 100 milliseconds to execute, or whatever we draw the line, to keep our fast brick tests fast enough to give us a really fast feedback loop. The project tests also give us a way to write tailor-made tests that are unique per project.
The second category is the brick tests. To keep the feedback loop short, we should only put fast-running tests in our bricks. This will give us a faster feedback loop, because the brick tests are the ones that are executed when we run poly test while the project tests are not.
But does that mean we are only allowed to put unit tests in our bricks? No. As long as the tests are fast (by e.g. using in-memory databases) they should be put in the bricks they belong to.
Before we continue, let's commit what we have done so far and mark the workspace as stable:
git add --all
git commit -m "Added tests"
git tag -f stable-lisa
If we execute the info command again:
...the * signs are now gone and nothing is marked to be tested.
The tool only executes tests if a brick is directly or indirectly changed. A way to force it to test all bricks is to pass in :all-bricks:
poly info :all-bricks
Now all the brick tests are marked to be executed, except for the development project. To include dev, also add :dev:
poly info :all-bricks :dev
To include all brick and project tests (except dev) we can type:
poly info :all
...to also include dev, type:
poly info :all :dev
Running the brick tests from the development projects are something we don't normally need to do, but it's good to know that it's supported.
Now let's see if it actually works:
poly test :all :dev
Projects to run tests from: command-line, development
​
Running tests for the command-line project using test runner: Polylith built-in clojure.test runner...
Running tests from the command-line project, including 2 bricks and 1 project: user, cli, command-line
​
Testing se.example.cli.core-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing se.example.user.interface-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing project.command-line.dummy_test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
Running tests for the development project using test runner: Polylith built-in clojure.test runner...
Running tests from the development project, including 2 bricks and 1 project: user, cli, command-line
​
Testing se.example.cli.core-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing se.example.user.interface-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Execution time: x seconds
Looks like it worked!
Test setup and teardown
Sometimes we need to perform some test setup/teardown before and after we execute the tests for a project.
If any code is used by more than one project, we can put it in a separate component, but in this case we should put it in the command-line project's test directory because it's not used by any other project.
Let's create a test-setup namespace in the project's test directory and add two functions to it:
example
├── projects
│   └── command-line
│       └── test
│           └── project
│               └──command_line
│                  └──test_setup.clj
(ns project.command-line.test-setup
  (:require [clojure.test :refer :all]))
​
(defn setup [project-name]
  (println (str "--- test setup for " project-name " ---")))
​
(defn teardown [project-name]
  (println (str "--- test teardown for " project-name " ---")))
We need to keep two things in mind:
Make sure the source code which contains our function, is accessible from the project it's executed from (the command-line project in this case). Here the project's own test directory was already added earlier by the create project command, so we are fine.
Make sure the functions take exactly one parameter, the project name.
We also need to specify the two functions in workspace.edn:
 ...
 :projects {"development" {:alias "dev"}
            "command-line" {:alias "cl"
                            :test {:setup-fn project.command-line.test-setup/setup
                                   :teardown-fn project.command-line.test-setup/teardown}}}}
If we don't need the tear-down function, we can leave it out.
Let's run our tests:
poly test :all
Projects to run tests from: command-line
​
Running test setup for the command-line project: project.command-line.test-setup/test-setup
--- test setup for command-line ---
​
Running tests for the command-line project using test runner: Polylith built-in clojure.test runner...
Running tests from the command-line project, including 2 bricks and 1 project: user, cli, command-line
​
Testing se.example.cli.core-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing se.example.user.interface-test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
​
Testing project.command-line.test-setup
​
Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
​
Test results: 0 passes, 0 failures, 0 errors.
​
Testing project.command-line.dummy_test
​
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
​
Test results: 1 passes, 0 failures, 0 errors.
Running test teardown for the command-line project: project.command-line.test-setup/test-teardown
--- test teardown for command-line ---
​
​
Execution time: 1 seconds
​
Nice, it worked!
Summary
Let's summarise the different ways to run the tests. The brick tests are executed from all projects they belong to except for the development project (if :dev is not passed in):
Command
Tests to execute
poly test
All brick tests that are directly or indirectly changed.
poly test :project
All brick tests that are directly or indirectly changed + tests for changed projects.
poly test :all‑bricks
All brick tests.
poly test :all
All brick tests + all project tests (except development), executed from all projects.
To also execute the brick tests from the development project, pass in :dev:
poly test :dev
All brick tests that are directly or indirectly changed, executed from all projects (development included).
poly test :project :dev
All brick tests that are directly or indirectly changed, executed from all projects (development included) + tests for changed projects (development included).
poly test :all‑bricks :dev
All brick tests, executed from all projects (development included).
poly test :all :dev
All brick tests, executed from all projects (development included) + all project tests (development included).
Projects can also be explicitly selected with e.g. project:proj1 or project:proj1:proj2.
We can also filter which bricks to run the tests for with e.g. brick:b1 or brick:b1:b2.
These arguments can also be passed into the info command, as we have done in the examples above, to get a view of which tests will be executed.
Finally, there is a way to restrict what test code to include for a project, by giving a list of bricks. This can be specified in workspace.edn, e.g.:
{...
 :projects {"development" {:alias "dev", :test []}
            "command-line" {:alias "cl", 
                            :test {:include ["cli]
                                   :setup-fn se.example.test-helper.interface/setup
                                   :teardown-fn se.example.test-helper.interface/teardown}}}}
...or by using this syntax:
{...
 :projects {"development" {:alias "dev", :test {:include []}
            "command-line" {:alias "cl", :test {:include ["cli"]}}}
If we run the info command with these settings:
poly info :all :dev
...the test source code will no longer be included in the development project, and only cli is included for the command-line project. This can be useful when we don't want to run the same brick tests from all our projects, as a way to get a faster test suit.
Note that if the tests directory for a brick is excluded from a project like this, they will never be tested from that project even if we pass in :all.
How tests are executed
Let's start with the development project. The main purpose of this project is to allow us to work with our code from an IDE using a single REPL. When doing that, the project must be set up in a way that it 100% compatible with tool.deps and the IDE integration. This is also the reason we have to add the test paths explicitly in ./deps.edn, which gives us access to the tests from the REPL.
To give us access to the src and resources paths from the REPL, we often add them as :extra-paths because we want to make sure that the IDE integration will work in all the development environments on the market.
Note: At the time of writing, adding bricks to development using the :local/root syntax works fine in VSCode/Calva and Emacs/CIDER, but unfortunately not in IDEA/Cursive, see this issue. However, if your organisation doesn't use Cursive, it should be fine to use the :local/root syntax even for the development project.
The ./deps.edn config file sets up all our paths and dependencies, and when we include the dev and test aliases (and sometimes profile aliases, described in the next section) we inform tools.deps what source code and libraries should be accessible from our IDE and REPL. When this is set up correctly, we are also able to run our tests from the REPL, which will have access to all test and src code. Libraries that are defined in the src context will therefore automatically be accessible when running the tests. Additional libraries that are only used from the tests should be defined in the test context.
When we run the test command, the tool will detect which components, bases and projects have been affected since the last stable point in time. Based on this information, it will go through all the affected projects, one at a time, and run the component, base, and project tests that are included in each project. 
This set of tests will be executed in isolation from its own classloader which will speed up the test execution and make it more reliable. Libraries from both the src and test context (and libraries that they depend on) will be used when the tests are executed. If :verbose is given when running the tests, the libraries and paths that are being used will be printed out. The development project can also be used to run tests, but that's not its main purpose.
If the projects A, B, C and D are included in the test run and if a test in B fails (or a project setup or teardown) then the whole test run will stop, and no tests are executed for C or D.
The libraries to use in each project when running the poly test command is the sum of all library dependencies that are defined in all the components and bases (either indirectly via local/root or directly by using :deps/extra-deps). If a library is defined more than once in the set of bricks and projects, then the latest version of that library will be used, if not overridden by :override-deps in the project.
At the project level we only need to define the libraries that are not defined in the included bricks (specified by its :deps key) which can be libraries like clojure itself, org.clojure/clojure, that we don't want to repeat in all our bricks.
If we have a brick like datomic-ions, we can specify which repository it needs, like this. We can verify that the repo is picked up by the brick by executing poly ws get:components:datomic-ions:maven-repos:
{"datomic-cloud" {:url "s3://datomic-releases-1fc2183a/maven/releases"}}
...and used by the invoicing project by executing poly ws get:projects:invoicing:maven-repos:
{"central" {:url "https://repo1.maven.org/maven2/"},
 "clojars" {:url "https://repo.clojars.org/"},
 "datomic-cloud" {:url "s3://datomic-releases-1fc2183a/maven/releases"}}
Every project that uses the datomic-ions brick will now also include the datomic-cloud repository.
Pluggable test runners
Starting from version 0.2.15, the poly tool supports test runners other than the built-in clojure.test runner by implementing the TestRunner.
At the time of writing, there is support for the Kaocha test runner.

Workflow - Previous

Flags

Next - Workflow

Profile

Last modified 1mo ago

Export as PDF

Copy link

On this page