Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts

Monday, December 30, 2013

Ignoring method invocations with RSpec message expectations

I often find myself with RSpec test examples that are verifying that log output meets expectations, but where the test is only concerned with verifying a single, specific log message has been generated. But usually the logger is receiving additional output that the test is not concerned about. Unfortunately, this output must still be "expected", lest the test will fail. There is (apparently) no way in RSpec to write an message expectation with specific arguments that should occur "at some point", when the message is also received at other times with different arguments.  For example this simple test example fails if logger.info is called more than once by the code being tested:

it "logs a specific message" do
  logger.should_receive(:info).with(/text that must be logged/)
  invoke_your_test_code()
end

RSpec will complain with something like:

expected: (/text that must be logged/)
     got: ("starting process...")


Writing and maintaining the code that explicitly expects all of the other logging calls is cumbersome to write and maintain, and makes the test fragile to log output changes that are not of concern to the test.

However, I found that one can gracefully ignore this extraneous log output by using the any_number_of_times expectation method in conjunction with the specific expectation to ignore all of the other method invocations, as follows:

it "logs a specific message message" do
  logger.should_receive(:info).with(/text that must be logged/)
  logger.should_receive(:info).any_number_of_times
  invoke_your_test_code()
end

The any_number_of_times expectation must occur after the specific expectation that is being verified; otherwise the expected log output will be "consumed" by the any_number_of_times expectation, and the test will fail.

Wednesday, June 30, 2010

Testing JSF backing beans

No matter what level of unit and integration testing is performed on one's code, there still seems to be no substitute for running tests directly from the user interface.  And while I've tried my hand at automating the UI tests for web applications, I find the task quite tedious and the scripts/code difficult to maintain.  So I find myself having to manually perform a battery of UI tests on my web application, enduring the dreaded Tomcat deploy/restart cycle (no snickering from the Ruby on Rails audience please!).

In particular, I find that the use of Hibernate and Spring's declarative transaction management makes it difficult to guarantee within my integration tests that entity objects are always being properly managed by the current Hibernate session.  That is, ensuring that LazyInitializationExceptions, NonUniqueObjectExceptions, etc., will not be thrown for a given call stack of UI backing bean and service methods.  This is difficult to test for "outside of the container", in a standalone integration test.  The complexity stems from the fact that backing bean methods (the entry points of UI actions) and the service layer methods that they call can each be transactional.  So the developer must guarantee that the entities being passed into a transactional method comply with the method's expectations for the "persistent", "detached", or "transient" state of these entities.  Things get really ugly when the domain model's save/update/persist cascades are different than what are needed to reattach all of the related entities that are needed by the method being called.

In my web application, I use Spring to instantiate all of the JSF backing bean objects, making use of the Spring-provided DelegatingVariableResolver.  (This allows the backing beans to be proxied and endowed with Spring's AOP functionality, for declarative transactions, logging aspects, etc.)  But testing backing beans with this design is made difficult by the fact that they must be instantiated by Spring within the test. This is solved by using AbstractDependencyInjectionSpringContextTests, which allows us to create a Spring context from which we can retrieve our backing beans.  However, this is still not enough, since in my case, the backing beans use "session" scope, and so normally require that they are instantiated within a servlet container, or at least that a FacesContext can be acquired.

To avoid the servlet container/FacesContext requirement, I figured out that I could create a MockSessionScope that can emulate a single extant servlet session, without JSF being initialized. This allows us to instantiate and retrieve our JSF backing bean objects from Spring, within our integration tests, and make calls on the backing bean methods with all AOP behavior enabled.  In this way, we can recreate the full call stack into our application, as if our JSF framework was calling the backing bean directly in response to an HTTP request.  And without using a browser client (real or headless).  Most significantly, we are now able to test the full transactional context that exists when our backing bean and service methods are called, allowing us to detect and debug problems merely by running our integration tests.  No more Tomcat deploy/restart/manually-testing cycle!

To make use of this we simply need to define a CustomScopeConfigurer in our testing spring context configuration that specifies the MockSessionScope for the "session" scope:

<bean
  class="org.springframework.beans.factory.config.CustomScopeConfigurer">
  <property name="scopes">
    <map>
      <entry key="session">
        <bean class="edu.harvard.med.screensaver.ui.MockSessionScope"/>
      </entry>
    </map>
  </property>
</bean>


The one thing that still is not exercised by this testing design is the rendering of the web pages, which can still be a source of problems.  Note that it is also necessary to ensure that the UI layer code being tested does depend upon a FacesContext or an HttpSession.  For the former, see various approaches suggested by others. For the latter, one can use Spring's MockHttpSession, as necessary.  The above design thus has some drawbacks, but we have at least raised the level of our tests one step closer to automated UI testing, and without the pain.

Thursday, July 9, 2009

continue; test; ant; scala

Well, having a baby and an arm injury in the last 8 months provides me with an acceptable excuse for not having posted an entry in some time.

I just added code coverage report support to my work project by integrating Cobertura into my project. I'm quite happy with the simplicity, ease, and usefulness of Cobertura. It looks like my project has 55% test code coverage, which I'm rather happy with, since this includes a considerable amount of untested UI code. (I've tried to make use of JSFUnit in the past, but the code-deploy-run cycle is entirely too burdensome--it really needs a run-time interpreted scripting language to make this an effective tool, IMO.). I also think I'm including the test code itself in the coverage result, which is not honest. I'll tweak things a bit, to get some accurate results, and repost my coverage stats. Most importantly, the Java packages that I deem to be most critical have very high test coverage (80-100%).

The vast majority of the time it took to accomplish this was consumed by trying to get Ant tasks to work. Boy, I really dislike Ant, and its lack of well-written documentation. I find the uptodate task docs are rather misleading, making one think you can compare respective files in parallel directories, but this either doesn't work at all, or I'm missing something. The nested element seems to hold the promise that one can do this, but in all cases, it seems you must compare to a single "flag" up-to-date file. Oh well.

I'm back to Scala, finally. Reading through the Artima book (printed version). Happy to see my name showed up in the acknowledgment section! (thanks to the feedback I provided on an early pre-print version) I hope to start using Scala initially by finding some scripting needs on my work project.