-
Notifications
You must be signed in to change notification settings - Fork 7.6k
drivers: stepper: tests: Test Generalization #85658
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First rough pass, thanks @jbehrensnx amazing effort!
@@ -6,4 +6,7 @@ config ENTROPY_GENERATOR | |||
bool "Shuffle the order of tests and suites" | |||
default y | |||
|
|||
config SPEED_STEPS | |||
int "Test speed in microsteps/s, also used as number of steps to make for certain tests" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a bit long and a grammar error, move this to a help text maybe?
config ENTROPY_GENERATOR | ||
depends on ZTEST_SHUFFLE | ||
bool "Shuffle the order of tests and suites" | ||
default y |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like its reverse - shuffling would depend on the entropy generator no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically yes, but it is in that order to ensure that ENTROPY_GENERATOR
is automatically selected when shuffling. ZTEST_SHUFFLE
having select ENTROPY_GENERATOR
is probably the better option
tests/drivers/stepper/stepper_api/boards/mimxrt1060_evk_mimxrt1062_qspi_B.overlay
Outdated
Show resolved
Hide resolved
@jilaypandya For some reason you were not included automatically |
6e98d0a
to
a2acc50
Compare
Thanks @jbehrensnx great work in unifying the tests :)
Should we not fix the stepper api specification before writing tests, i mean it's probably a change to the API documentation, or? |
On some level you could say yes. The tests can however help us find cases where the current api specification is insufficient. That said, I would agree that we should complete the specification discussion before merging/approving this PR. The other question is, should we discuss the api specification in this PR or in a RFC? |
a2acc50
to
d80586e
Compare
|
Yes a RFC sounds good 👍 |
Please rebase :) |
d80586e
to
fb71476
Compare
Significant rework, incorporating the changes that have occurred since. Replaced test_config.yaml with tags. Usage of alias instead of chosen, as that is what the stepper_api test suite was using. |
tests/drivers/stepper/stepper_api/boards/mimxrt1060_evk_mimxrt1062_qspi_B/allegro_a4979.overlay
Outdated
Show resolved
Hide resolved
Mimxrt1060-EVKB | ||
--------------- | ||
|
||
NOTE: For this test no real a4979 was used and thus the following pin configuration is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In overlays via comments we can and do mention the pinout. Please drop this and other tables
NOTE: For this test no real a4979 was used and thus the following pin configuration is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While i do appreciate this extensive testing, for an api that is still experimental this would mean each time a change is introduced, quite a bit of refactoring will have to be undetaken
int "Test speed in microsteps/s" | ||
default 50 | ||
help | ||
Besides being the test speed, this value is also used in several tests as the number of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Description says otherwise
...rs/stepper/stepper_api/boards/mimxrt1060_evk_mimxrt1062_qspi_B/adi_tmc2209_workqueue.overlay
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have the moving of files in a separate commit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can see what I can do, but as squashing everything into one commit was not the last thing I did, it might be difficult.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right now lot of different things are squashed into one commit. Let's bring the changes step by step in.
In the first step probably just transfer the existing drv8424_api tests to stepper_api. That would also make reviewing easier and give more context about what is happening.
tests/drivers/stepper/stepper_api/boards/nucleo_f767zi/adi_tmc2209.overlay
Outdated
Show resolved
Hide resolved
tests/drivers/stepper/stepper_api/boards/nucleo_f767zi/ti_drv8424.overlay
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there are some overlays which you have not actually tested on the hardware, can we please not include them? I stick to the example of nucleo_g071rb
since i test few drivers that i have with that board.
Also this leaves room for further contributors who want to get their first PR badge :-)
Nucleo F767ZI
-------------
NOTE: For this test no real tmc2209 was used and thus the following pin configuration is
theoretical.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this actually tested on the hardware or is it just suggestive
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The drv8424, tmc2209 and a4979 drivers dont actually need hardware at this point. One can simply define and configure a node for them and then run the tests successfully. Which is what I did for the tmc and allegro.
I defined devicetree nodes for the drivers and tested them on the two boards. The wording is admittedly maybe a bit misleading, as I tested them on actual boards, just without an actual stepper controller.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is basically the same as the native_sim tests, just on a real board and not a simulated one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the readme so that this is hopefully clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this some pre-work for the upcoming ramp drivers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More or less, plus the tmc50xx driver already exists. As we have drivers with differing movement behavior, separating these tests seemed like a good idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so set_micro_step_interval
is the only way of setting speed in stepper api. tmc50xx in its current form does not support this, the timing of the steps is decided internally by the IC. One can set the ramp parameters and the rest of the work is taken over by the IC.
so stepper api inherently does not have any functions which would suggest anything other than a fixed step interval ticking i.e. set_micro_step_interval
. Hence there is no need to diverge into two different test-suites as of now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually there is. stepper_api
is intended to include tests that all stepper drivers need to be able to pass. This includes tmc50xx. The remaining functionality needs to be tested in other testsuites. This is where stepper_const_velocity
comes into play. Another test suite concerning the specific behavior of tmc50xx should also be developed at some point, but that is beyond the scope of this PR.
extra_args: | ||
- platform:native_sim/native/64:DTC_OVERLAY_FILE="boards/native_sim/allegro_a4979.overlay" | ||
- platform:nucleo_f767zi/stm32f767xx:DTC_OVERLAY_FILE="boards/nucleo_f767zi/allegro_a4979.overlay" | ||
- platform:mimxrt1060_evk@B/mimxrt1062/qspi:DTC_OVERLAY_FILE="boards/mimxrt1060_evk_mimxrt1062_qspi_B/allegro_a4979.overlay" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please only add the overlays if they are actually tested on the hardware
NOTE: For this test no real a4979 was used and thus the following pin configuration is
theoretical.
.callback = stepper_print_event_callback, | ||
.callback = stepper_api_print_event_callback, | ||
.base_ms_resolution = CONFIG_BASE_MS_RESOLUTION, | ||
.test_ms_resolution = CONFIG_TEST_MS_RESOLUTION, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where exactly is this getting used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
During the rework I forgot to add the corresponding test.
Edit: Added missing test.
int32_t steps = 100; | ||
bool moving = true; | ||
|
||
(void)stepper_set_microstep_interval(fixture->dev, fixture->test_interval); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(void)stepper_set_microstep_interval(fixture->dev, fixture->test_interval); | |
(void)stepper_set_microstep_interval(fixture->dev, CONFIG_TEST_MS_RESOLUTION); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That constant is intended to be used to test that a new microstep resolution can be successfully set. I forgot to add the test during the rework, which will be fixed. Managing the interval via a constant is however something I will do.
(void)stepper_move_by(fixture->dev, steps); | ||
(void)stepper_enable(fixture->dev, false); | ||
(void)stepper_is_moving(fixture->dev, &moving); | ||
zassert_false(moving, "Driver should not be in state is_moving after being disabled"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While this statement is indeed correct, its just a bit handful to read :)
zassert_false(moving, "Driver should not be in state is_moving after being disabled"); | |
zassert_false(moving, "stepper should not be moving once disabled"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your recommendation is somewhat imprecise. There is a difference between the stepper being in state is_moving
versus actually moving (i.e. position counter changing). The new error message could indicate the latter, which is however tested in another test.
(void)stepper_enable(fixture->dev, true); | ||
(void)k_msleep(100); | ||
(void)stepper_is_moving(fixture->dev, &moving); | ||
zassert_false(moving, "Driver should not be in state is_moving after being reenabled"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
zassert_false(moving, "Driver should not be in state is_moving after being reenabled"); | |
zassert_false(moving, "Stepper should not start moving upon being just enabled"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Being reenabled is actually an important part of the test and thus should occur in the error message.
I will drop the qemu and native sim platforms from the |
That is actually a very good reason to write all these tests, as they will help to ensure, that after these changes the api still behaves as expected. It is not quite Test Driven Development, but it borrows at least a few ideas from it. |
fb71476
to
60dfb91
Compare
0b73f17
to
b43788e
Compare
{ | ||
int32_t steps = 1000; | ||
int32_t position_1; | ||
int32_t position_2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
int32_t position_before_reenabled;
int32_t position_after_reenabled;
################# | ||
|
||
Tests the general functionality and behaviour of stepper drivers to ensure conformity to the stepper | ||
api specification. All new stepper drivers should pass this test suite. It should be noted, that this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All new stepper drivers should pass this test suite.
This is actually a process decision, tbh i have not seen this in any other driver subsystem till now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am aware of this. That said, having a test suite that every new driver should pass (excluding exceptional circumstances with a good reason, I am not ruling these out) would improve the quality and consistency of new stepper drivers.
There are multiple possible reasons why other driver subsystems have not done this. One of them is finding a common core of functionality to set as a criteria (which can be seen even with stepper drivers, considering I split it in 2 test suites). The counter drivers for example have an extensive test suite, but it is missing tests for some functions while having tests not every driver can pass.
Another reason is that it might be very difficult to test these drivers in a meaningful way.
Then there the large number of existing drivers that might fail, which would result in many changes to these. In that regard, the low number of stepper drivers is a boon, as it makes this quite manageable.
And finally, people might not have want/been able to spend the required resources on this.
Another thing to consider is that with the growing commercial importance of Zephyr, testing will become more important and the stepper api could get a headstart on this.
Also, the goal of these tests is to test that a driver fulfills the api specification. If a new driver fails these tests, it does not fulfill the specification, so why would we want to accept it?
If we do it this way, there will be followup discussions about issues we find and about clarifications, but I think it is an important first step.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And finally, people might not have want/been able to spend the required resources on this.
This is a very good reason to not increase the entry bar for new drivers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And finally, people might not have want/been able to spend the required resources on this.
This is a very good reason to not increase the entry bar for new drivers.
This was in reference to spending the resources creating such a test suite.
Making the test suite mandatory doesn't really increase the entry bar for new drivers noticeably. In fact, one could argue that it lowers it. When developing a new driver you want to use the test suite during development anyway, as it is an easy way to check that your driver is doing the correct things. One will still need to use the shell on occasion, but the tests make the development process much easier. And with this background, succeeding the test suite becomes a normal part of developing a driver. Thus not raising the entry bar.
In the end, we have something akin to Test Driven Development, but with the tests already existing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am aware of this. That said, having a test suite that every new driver should pass (excluding exceptional circumstances with a good reason, I am not ruling these out)
What would be these exceptional circumstances?
I have mentioned this in one of the earlier comments as well, testing the purely gpio-based drivers is easy and as far as i can see can be very well covered by such test suites. But drivers like tmc5xxx or for that matter even tmc2209(only uart based) would need some sort of emulator. So what would we do in this case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would be these exceptional circumstances?
I'm not sure until it happens. As these tests are not perfect, there is always the possibility that there is some edge case or another reason that I did not consider In that case, these will need to be discussed. That is what I meant to say with this.
I have mentioned this in one of the earlier comments as well, testing the purely gpio-based drivers is easy and as far as i can see can be very well covered by such test suites. But drivers like tmc5xxx or for that matter even tmc2209(only uart based) would need some sort of emulator. So what would we do in this case?
Those that can be tested via native_sim/qemu
are, the rest can't be tested this way. That is a limitation of Zephyr, that some things can only be tested on real hardware. If however, someone develops a new driver, they have the means to test it. In that case we either have to trust their word or ask for the test log. Its one of the reasons, why only new tests should need to pass the test suite, as at that point there is somebody that can test the driver, later on that is however no longer the case.
The ideal solution in these cases would be for someone to develop an emulator, or at least stubbs for testing under native_sim/qemu
but that is a lot of work and would, as you mentioned, raise the bar for entry, which is why I consider that nice to have but not required (and there is an argument to be made, that these emulators should be developed by the hardware developers, but that is a whole other discussion).
step-gpios = <&arduino_header 10 0>; /* D4 */ | ||
en-gpios = <&arduino_header 11 0>; /* D5 */ | ||
msx-gpios = <&arduino_header 12 GPIO_ACTIVE_HIGH>, /* D6 */ | ||
<&arduino_header 13 GPIO_ACTIVE_HIGH>; /* D7 */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
msx-gpios = <&arduino_header 12 GPIO_ACTIVE_HIGH>, /* D6 */
<&arduino_header 13 GPIO_ACTIVE_HIGH>; /* D7 */
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the upside of adding these overlays for so many different boards?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Partly they are the result of me testing these drivers on these boards too see how they perform. The other reason is that counters and possibly the working queue can behave slightly different between boards. And the step-dir drivers sometimes use the step-dir implementation in slightly different ways. Thus, testing drivers on many different boards can improve the chance of finding errors.
This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time. |
Adds a check for 0 steps to take for move_by in both step_dir_stepper_common and the h-bridge driver. Signed-off-by: Jan Behrens <[email protected]>
Next attempt. Moved all tests from This is still work in progress. One addition is the Also, before somebody asks, I removed the |
b43788e
to
4e55a8e
Compare
{ | ||
int32_t pos = 10u; | ||
int ret; | ||
(void)stepper_enable(fixture->dev); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think, a better check of these calls is necessary, rather than void-ing out the returns. You can do something like:
zassert_equal(stepper_enable(fixture->dev), 0, "Failed to enable device");
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it is still necessary. Enable only charges the coils and has no impact on the stepper behavior otherwise. That said, a dedicated test for checking enable/disable would be an option. Checking it in every test case would be overkill imo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you justify, why asserting the function call is not necessary? You would never know if the function call was erroneous if you void the return.
As written above, since one of the recent changes to the stepper api, the enable/disable functions no longer influence the step-logic. Thus their success or failure is irrelevant. Enable/disable is currently used because it was previously necessary and they would appear in a workflow on physical hardware, but I am currently wondering if it would be better to just remove them entirely.
@jilaypandya What is your opinion, should the enable/disable function calls be just removed? They no longer fulfill the functionality they previously did.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One should always assert the return values of function calls in tests.
you can do something like this
zassert_not_equal(stepper_enable(fixture->dev), -EIO, "Failed to enable device");
, coz if the enable returns -ENOTSUP
or 0, the test-suite shall pass.
you can move enable in before, so that you don't have to repeat it in each and every test.
@dipakgmx thanks for pointing this out, i will make sure i do this as well.
4e55a8e
to
af9bf00
Compare
The If a physical board is still a no-go, then allowing a single emulation platform (maybe EDIT: Edited |
|
af9bf00
to
431afe0
Compare
Removed the test filtering and improved the range check for the run tests. I also removed the |
@@ -18,6 +19,8 @@ struct k_poll_signal stepper_signal; | |||
struct k_poll_event stepper_event; | |||
void *user_data_received; | |||
|
|||
#define TEST_STEP_INTERVAL (20 * USEC_PER_SEC) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has to be a Kconfig Configuration Option
{ | ||
int32_t pos = 10u; | ||
int ret; | ||
(void)stepper_enable(fixture->dev); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One should always assert the return values of function calls in tests.
you can do something like this
zassert_not_equal(stepper_enable(fixture->dev), -EIO, "Failed to enable device");
, coz if the enable returns -ENOTSUP
or 0, the test-suite shall pass.
you can move enable in before, so that you don't have to repeat it in each and every test.
@dipakgmx thanks for pointing this out, i will make sure i do this as well.
431afe0
to
977160c
Compare
ZTEST_F(stepper, test_set_micro_step_res_valid) | ||
{ | ||
int ret = | ||
stepper_set_micro_step_res(fixture->dev, CONFIG_STEPPER_TEST_MICROSTEP_RESOLUTION); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks a bit strange. Most probably happened because of clang-format.
Adjust the code as following.
int ret;
ret = stepper_set_micro_step_res(fixture->dev, CONFIG_STEPPER_TEST_MICROSTEP_RESOLUTION);
} | ||
|
||
ZTEST_F(stepper, test_target_position_w_fixed_step_interval) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was this test similiar to a test in drv84xx test suite, and hence been deleted?
Please add the tests from drv84xx to stepper_api in a commit and then delete the duplicate test cases and do corresponding adjustments to stepper_api test suite in another commit. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved the removal to the drv84xx removal commit and renamed it to clarify that it removes superfluous tests.
Separating modifications to the tests into a separate commit would be very difficult at this point, especially, as some changes are needed to run the tests. That said, I have adjusted the commit message to indicate that there also some modifications to tests.
{ | ||
int32_t pos = 10u; | ||
int ret; | ||
(void)stepper_set_microstep_interval(fixture->dev, CONFIG_STEPPER_TEST_MICROSTEP_INTERVAL); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One of the previous review comments about asserting the return values of stepper_api functions needs to be addressed. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering how often this function is called, I moved it into the before
function and added a separate test for this. Because these tests are only for non-acceleration drivers, this should suffice.
977160c
to
559b675
Compare
Moves the tests of the drv84xx/api test suite into the stepper_api test suite. To facilitate this and for cleanup purposes, several tests are modified. Signed-off-by: Jan Behrens <[email protected]>
559b675
to
35ea2df
Compare
Removes the drv84xx/api test suite as well as a duplicate test in stepper_api. Signed-off-by: Jan Behrens <[email protected]>
35ea2df
to
7ead59e
Compare
|
This PR includes the test generalization discussed in PR #82843.
This PR has 3 objectives, first to offer a set of test suites for stepper drivers that can be used to help develop new drivers. The second objective is to offer a specification of the expected API behavior, something that is currently missing. And thirdly, to offer a tool to evaluate new drivers before approval.
While several stepper drivers have been developed, testing was often done only haphazardly, primarily using the stepper shell, such as PR #79120. Testing via the shell is not only inconvenient and time consuming, it is also unreliable and non-repeatable, leading to many bugs and other issues to slip through. This set of test suites counteracts this by offering a set of repeatable tests that cover a wide range of expected behavior, making development easier and improving driver quality.
The stepper API currently lacks a specification for expected behavior. This not only makes driver development more difficult, as it is unclear how the new driver should behave in certain situations, it also leads to differences in behavior between drivers. This is in stark contrast to the goals of an API, where the user shouldn't care about the specific implementation and instead each api call should lead to the same result. These tests can alleviate this by defining expected behavior. Drivers that successfully pass them behave the same in the covered areas, improving their interchangeability from the user perspective.
Thirdly, does these test suites offer a way to evaluate new drivers before approval and thus ensure a certain level quality. Ideally, a new stepper should pass all applicable tests before it is approved.
Test Suites
stepper_api
This is a complete replacement of the existing stepper_api test suite. It checks general behavior of the driver and the results of incorrect input.
stepper_const_velocity
This test suite checks that drivers that don't use acceleration and thus run at a constant velocity, move as expected. This includes that
move_to
andmove_by
finish in the expected time, and that all three movement commands result in the expected position after a given time.Current Test Results
All Tests were executed on a Nucleo F767ZI board with the
CONFIG_ZTEST_SHUFFLE
flag set, ensuring both random order and repetition. For the GPIO Stepper Controller, incorrect spinlock usage results intest_micro_step_res_set_incorrect_value_detection
to crash the system. For this driver the test is skipped and considered to fail. The following table gives the test pass rate for both test suites and all tested drivers.Limitations
Like all tests, this set of test suites has its limitations. While the new
stepper_api
test suite should be applicable to all stepper driver, thestepper_const_velocity
, as the name suggests, is only meant for drivers that move at a constant velocity. Drivers that use acceleration, such as tmc50xx or the acceleration step-dir implementation currently under development require their own test suite to evaluate movement.Another limitation of this set of test suites is that they are limited to evaluating api return values. They can not check the physical behavior. In the case of a step-dir driver for example, they can not check, if the order is given for 100 steps, that the step signal actually send 100 pulses and that they occurred with the right frequency. Twisters support for these kinds of tests is currently limited. In addition, they require a far greater degree of customization for the target driver, going beyond the scope of this PR.
Possible Future Work