Assert a method is not called using Mox

Let's say you want to unit test a function and make sure that another function is not called at any point during the routine.

To do it, simply stub out the function you want to assert isn't called, and do not record a call to it before putting the mocks into replay mode.

Here's some example code:

import unittest

import mox


class Mystery(object):
    def want(self):
        pass

    def do_not_want(self):
        pass

    def func(self):
        self.want()


class TestIt(unittest.TestCase):
    def test(self):
        mox = mox.Mox()
        
        m = Mystery()

        # stub out the functions you're interested in verifying
        mox.StubOutWithMock(m, 'want')
        mox.StubOutWithMock(m, 'do_not_want')

        # record calls to stubbed out functions that are
        # expected to be called, in the order expected,
        # as many times as expected
        m.want()

        # set the mocks to replay mode
        mox.ReplayAll()

        # call the function you want to unit test
        m.func()

        # verify the expected calls were made, and the
        # unexpected calls were not made
        mox.VerifyAll()

if __name__ == '__main__':
    unittest.main()

The above code yields the result:

.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

If we update class Mystery to:

class Mystery(object):
    def want(self):
        pass

    def do_not_want(self):
        pass

    def func(self):
        self.want()
        self.do_not_want()

We get this result instead:

F
======================================================================
FAIL: test (__main__.TestIt)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "notcalled.py", line 28, in test
    m.func()
  File "notcalled.py", line 14, in func
    self.do_not_want()
  File "/Library/Python/2.6/site-packages/mox.py", line 765, in __call__
    return mock_method(*params, **named_params)
  File "/Library/Python/2.6/site-packages/mox.py", line 1002, in __call__
    expected_method = self._VerifyMethodCall()
  File "/Library/Python/2.6/site-packages/mox.py", line 1049, in _VerifyMethodCall
    expected = self._PopNextMethod()
  File "/Library/Python/2.6/site-packages/mox.py", line 1035, in _PopNextMethod
    raise UnexpectedMethodCallError(self, None)
UnexpectedMethodCallError: Unexpected method call do_not_want.__call__() -> None

----------------------------------------------------------------------
Ran 1 test in 0.008s

FAILED (failures=1)

Mock a socket timeout with HTTPretty

Let's say you have a client that raises MyClientTimeoutException if the underlying socket connect times out. You can unit test this by using the HTTPretty library and stubbing out socket. This example uses Mox.

import socket
import unittest

from httpretty import HTTPretty, httprettified
import stubout

class Test(unittest.TestCase):
    def setUp(self):
        super(Test, self).setUp()
        self.stubs = stubout.StubOutForTesting()

    def tearDown(self):
        super(Test, self).tearDown()
        self.stubs.UnsetAll()
        self.stubs.SmartUnsetAll()

    @httprettified
    def test_timeout_call(self):
        client = MyClient()
        HTTPretty.register_uri(HTTPretty.GET, "http://foo/")

        # fake a timeout
        def fake_create_connection(address, timeout):
            raise socket.timeout('timeout')

        self.stubs.Set(socket, 'create_connection', fake_create_connection)
        self.assertRaises(MyClientTimeoutException,
                          client.call,
                          "http://foo/")

Python Slicing and Copies

I've seen concern a number of times, on blogs or in code reviews, that slicing lists in Python negatively impacts performance, especially when dealing with large data. The concern is about objects being copied as a result of slicing.

I hadn't heard of deep copy being the default behavior for, well, anything in any language. So, it surprised me to think Python would behave like that and I started looking around for information. I could have done my own test initially instead, but I'm relatively new to Python and didn't immediately know how.

It took awhile to find anything. It's one of those things where probably most people know it, but those who don't aren't corrected often.

Anyway, I found in the introductory documentation on lists that "... slice operations return a new list containing the requested elements. This means that the following slice returns a shallow copy of the list..." A shallow copy means that only object references are copied. Strings behave the same way when sliced, probably because lists and strings are both sequence types.

Now, to an example I could have used instead of searching for documentation:

class String(object):
    def __init__(self, value):
        self.value = value

    def __repr__(self):
        a = []
        for c in self.value:
            a.append('%c [%s]' % (c, hex(id(c))))
        return ''.join([self.value, ': ', repr(a)])

s0 = String('OHAY')
print s0
s1 = String(s0.value[:1])
print s1
s2 = String(s0.value[1:])
print s2

And the output:

OHAY: ['O [0x100453a80]', 'H [0x1004d0780]', 'A [0x1004d07b0]', 'Y [0x1004d0840]']
O: ['O [0x100453a80]']
HAY: ['H [0x1004d0780]', 'A [0x1004d07b0]', 'Y [0x1004d0840]']

You can see from the object identities that only the references are copied, not the objects. This is generally inexpensive, that is, it's unlikely to hurt overall performance. Always measure first before trying to optimize.

Labels Containing Symbols are Not Searchable

I don't know how it took me this long to notice, but here on blogger.com, labels containing symbols are broken in that they can't be searched. I have used the label "c++" in the past for a couple of my posts and today when I clicked the search link for the "c++" label, it gave the message "No posts with label c++." instead of returning the labeled posts. I have changed to use "cplusplus" instead until there's hopefully a fix.

It took me a little while to find where I could report a bug and I opened this one.

Here's the link for reporting problems and bugs if you have any.

JNI Example with Class Instantiation (C++)

Awhile back I had to create a JNI wrapper for a C++ class that uses a proprietary core C API. Rewriting everything including the core API in Java would be safer and perform better but in this case not duplicating code was more important. I couldn't find a clear code example containing everything that was needed to 1) make the JNI work at all and 2) wrap a class and call its non-static member function, so I'm putting this here in case it might help someone else.

There are two source files you'll need to write:

1. A Java class that declares native methods, loads the library built using the JNI, and calls native methods.
   
2. JNI code that implements the native methods declared in the Java class and uses the C++ class you're wrapping. You'll build a library with this.
   

The JNI code file will need to #include a file you'll generate using javah, the C Header and Stub File Generator. The javah tool will create the JNI function prototypes that match the native methods you declared in the Java class.

Here is an example, A.java:

package com.company.product.component;
import java.util.Properties;

public class A {
    private long ptr_;

    private native long createA(long value);
    private native void destroyA(long ptr);
    private native long getResult(long ptr, byte[] input, long length);

    static {
        System.loadLibrary("AJni");
    }
    
    public A(long value) {
        ptr_ = createA(value);
    }
    
    /**
     * Destroy the A instance created by the JNI.
     * This must be called when finished using the A.
     */
    public void destroy() {
        destroyA(ptr_);
    }

    /**
     * Do something.
     *
     * @param input The input array
     * @param length The length of the input array to process
     * @return The result
     */
    public long getResult(byte[] input, long length) {
        return getResult(ptr_, input, length);
    }
}

There are three native methods declared in A.java: one that will create an instance of the C++ class, one that calls a member function of the class, and one that destroys the instance. There is a member variable, ptr_, which stores the pointer to the A instance that's created outside the JVM. You need this to call the member function, getResult(). You also need it to destroy the A instance when you're done. Once you have your Java class written you'll need to:

1. Use javac to compile A.java into A.class.
   
2. Run javah on the Java class like this: javah com.company.product.component.A
   to generate com_company_product_component_A.h which you'll include in the JNI code.
   

This is what com_company_product_component_A.h looks like:

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_company_product_component_A */

#ifndef _Included_com_company_product_component_A
#define _Included_com_company_product_component_A
#ifdef __cplusplus
extern "C" {
#endif

/*
 * Class:     com_company_product_component_A
 * Method:    createA
 * Signature: (J)J
 */
JNIEXPORT jlong JNICALL Java_com_company_product_component_A_createA
  (JNIEnv *, jobject, jlong);

/*
 * Class:     com_company_product_component_A
 * Method:    destroyA
 * Signature: (J)V
 */
JNIEXPORT void JNICALL Java_com_company_product_component_A_destroyA
  (JNIEnv *, jobject, jlong);

/*
 * Class:     com_company_product_component_A
 * Method:    getResult
 * Signature: (J[BJ)J
 */
JNIEXPORT jlong JNICALL Java_com_company_product_component_A_getResult
  (JNIEnv *, jobject, jlong, jbyteArray, jlong);

#ifdef __cplusplus
}
#endif
#endif

The header contains function prototypes that you'll need to provide implementations for in a C++ source file.

Here is the example JNI code, A.cc:

#include "com_company_product_component_A.h"
#include <A.hh>

JNIEXPORT jlong JNICALL Java_com_company_product_component_createA
    (JNIEnv *env, jobject obj, jlong value)
{
    return reinterpret_cast<jlong>(new A(value));
}

JNIEXPORT void JNICALL Java_com_company_product_component_destroyA
    (JNIEnv *env, jobject obj, jlong ptr)
{
    A *a = reinterpret_cast<A*>(ptr);
    if(a) {
        delete a;
    }
}

JNIEXPORT jlong JNICALL Java_com_company_product_component_getResult
    (JNIEnv *env, jobject obj, jlong ptr, jbyteArray input, jlong length)
{
    jbyte *inputArray = env->GetByteArrayElements(input, NULL);
    jlong result = reinterpret_cast<A*>(ptr)->
        getResult(reinterpret_cast<const char*>(inputArray), length);
    env->ReleaseByteArrayElements(input, inputArray, 0);
    return result;
}

The code includes A.hh, the header for the C++ class you're wrapping. All the use of the C++ class is here. Then, A.cc needs to be compiled and linked with the library you're wrapping into a new library, for example libAJni.so, that gets loaded by the Java code using System.load() as you saw in A.java earlier. That's pretty much it. You can now use the Java class A to interact with the C++ library libAJni.so which interacts with the C++ library you wanted to wrap in the first place.

A.class --> libAJni.so --> libA.so

Alternation without Capture/Extraction/Selection

This drove me crazy for longer than I wanted (since I would want that for varying amounts of time) so I will note it here for other frustrated people to find.

I had a regular expression from which I wanted to capture part of it in $1. I also had an alternation in it that needed grouping with parentheses. It kept capturing the alteration in $1 when I didn't care or want to capture that at all.

This is an example of what I had at first that didn't do what I wanted:

$string =~ /bytes\s+=\s+\d+\.?\d*(K|M)?\s+\(\s*(\d+\.?\d*)\%/;

The above code was giving me either 'K' or 'M' in $1 if either were there instead of what I wanted which was the second grouping (\d+\.?\d*). I just needed to know how to stop the capture since stuff like K|M? and K?|M? without parentheses didn't work right either.

After a lot of online searching using probably the wrong query terms, I found the concept of "non-capturing groupings" which are apparently denoted by (?:regex).

Changed my code to this to finally get what I wanted:

$string =~ /bytes\s+=\s+\d+\.?\d*(?:K|M)?\s+\(\s*(\d+\.?\d*)\%/;

This way the 'K' or 'M' isn't captured and I get the second grouping (\d+\.?\d*) stored in $1.

Storing Options with Multiple Values in a Hash using Getopt::Long

I'm relatively new to Perl and was writing a program that takes long options on the command line. I quickly found the Getopt::Long module and started out getting options where each option's value was stored in a separate variable. One of the options has multiple values, e.g. --option5 0 1 2.

When I started, I had something like:

my ($option1, $option2, $option3, $option4, @option5);
my $result = GetOptions('option1=s'    => \$option1,
                        'option2=i'    => \$option2,
                        'option3=s'    => \$option3,
                        'option4=s'    => \$option4,
                        'option5=i{,}' => \@option5);

I wanted to change it to store the options in a hash since I had several options but got confused with the option that takes multiple values using a repeat specifier. The reason was stated in the documentation that "The destination for the option must be an array or array reference." What I didn't realize is that I needed to use this syntax "Alternatively, you can specify that the option can have multiple values by adding a "@", and pass a scalar reference as the destination" since my destination was now an uninitialized hash entry.

I ended up with this:

my %options;
my $result = GetOptions(\%options, 'option1=s',
                                   'option2=i',
                                   'option3=s',
                                   'option4=s',
                                   'option5=i@{,}');