Writing Great Unit Tests: Why Bother?

Writing Great Unit Tests: Why Bother?

Unit testing is the practice of testing the components of a program automatically, using a test program to provide inputs to each component and check the outputs. The tests are usually written by the same programmers as the software being tested, either before or at the same time as the rest of the software. What’s the difference between a good unit test and a bad one? How do you learn how to write good unit tests? It’s far from obvious. Even if you’re a brilliant coder with decades of experience, your existing knowledge and habits won’t automatically lead you to write good unit tests, because it’s a different kind of coding and most people start with unhelpful false assumptions about what unit tests are supposed to achieve. It’s overwhelmingly easy to write bad unit tests that add very little value to a project while inflating the cost of code changes astronomically. Does that sound agile to you?

Unit testing is not about finding Defects

Unit tests, by definition, examine each unit of your code separately. But when your application is run for real, all those units have to work together, and the whole is more complex and subtle than the sum of its independently-tested parts. At a high-level, unit testing refers to the practice of testing certain functions and areas – or units – of our code. This gives us the ability to verify that our functions work as expected. That is to say that for any function and given a set of inputs, we can determine if the function is returning the proper values and will gracefully handle failures during the course of execution should invalid input be provided.

Ultimately, this helps us to identify failures in our algorithms and/or logic to help improve the quality of the code that composes a certain function. As you begin to write more and more tests, you end up creating a suite of tests that you can run at any time during development to continually verify the quality of your work.

A second advantage to approaching development from a unit testing perspective is that you'll likely be writing code that is easy to test. Since unit testing requires that your code be easily testable, it means that your code must support this particular type of evaluation. As such, you're more likely to have a higher number of smaller, more focused functions that provide a single operation on a set of data rather than large functions performing a number of different operations.

A third advantage for writing solid unit tests and well-tested code is that you can prevent future changes from breaking functionality. Since you're testing your code as you introduce your functionality, you're going to begin developing a suite of test cases that can be run each time you work on your logic. When a failure happens, you know that you have something to address.

Well then, if unit testing isn’t about finding bugs, what is it about?

I bet you’ve heard the answer a hundred times already, but since the testing misconception stubbornly hangs on in developers’ minds, per principle. As Test plan development forum keep saying, “TDD is a design process, not a testing process”. Let me elaborate: TDD is a robust way of designing software components (“units”) interactively so that their behavior is specified through unit tests. That’s all!

Tips for writing great unit tests

Enough theoretical discussion – time for some practical advice. Here’s some guidance for writing unit tests that sit comfortably at Sweet Spot on the preceding scale, and are virtuous in other ways too.

·         Make each test orthogonal (i.e., independent) to all the others

Any given behavior should be specified in one and only one test. Otherwise if you later change that behavior, you’ll have to change multiple tests. The corollaries of this rule include

·         Test passes but not testing the actual feature

Beware of such test cases making place in your code repository which seems to doing lots of stuff in code, but in actual they were doing nothing. They were sending requests to server and no matter what server respond, they were passing. Horror!! They will become liability on you to carry them, build them and running them every time but without adding any value.

·         Testing irrelevant things

This one is another sign of bad test case. I have seen developers checking multiple irrelevant things so that code passes with doing SOMETHING, well not necessarily the correct thing. Best approach is to follow single responsibility principle, which says, one unit test case should test only one thing and that’s all.

·         Don’t make unnecessary assertions

Which specific behavior are you testing? It’s counterproductive to Assert () anything that’s also asserted by another test: it just increases the frequency of pointless failures without improving unit test coverage one bit. This also applies to unnecessary Verify () calls – if it isn’t the core behavior under test, then stop making observations about it! Sometimes, Testing folks express this by saying “have only one logical assertion per test”. Remember, unit tests are a design specification of how a certain behavior should work, not a list of observations of everything the code happens to do.

·         Don't ship code with tests that fail, even if it doesn't matter that they fail.

It's not uncommon, particularly in test-driven development, to change your mind during design about which tests are correct or relevant, or to make an initial implementation that only covers some of the test suite. But that means you end up with failed tests that you don't actually care about. Remove them, or at very least, document them: anyone running your tests should be able to assume that a failed test indicates broken code.

·         Consider using a Code coverage tool to check how much of your code is actually being tested. Coverage doesn't tell you everything: it only measures static execution paths, but it can give you some idea of things you might have missed altogether.

·         Whenever you find a bug in “finished code”, add a test for it.

Make sure the test fails in the buggy code and passes when it is fixed. Areas of code you've found bugs in are more likely to be fragile in general, and bugs that have already been found are relatively highly likely to reappear.

·         Test only one code unit at a time

Your architecture must support testing units (i.e., classes or very small groups of classes) independently, not all chained together. Otherwise, you have lots of overlap between tests, so changes to one unit can cascade outwards and cause failures everywhere.

If you can’t do this, then your architecture is limiting your work’s quality – consider using Inversion of Control.

·         Separate out all external services and state

Otherwise, behavior in those external services overlaps multiple tests, and state data means that different unit tests can influence each other’s outcome. You’ve definitely taken a wrong turn if you have to run your tests in a specific order, or if they only work when your database or network connection is active. (By the way, sometimes your architecture might mean your code touches static variables during unit tests. Avoid this if you can, but if you can’t, at least make sure each test resets the relevant statics to a known state before it runs.)

·         Avoid unnecessary preconditions

Avoid having common setup code that runs at the beginning of lots of unrelated tests. Otherwise, it’s unclear what assumptions each test relies on, and indicates that you’re not testing just a single unit. An exception: Sometimes I find it useful to have a common setup method shared by a very small number of unit tests (a handful at the most) but only if all those tests require all of those preconditions. This is related to the context-specification unit testing pattern, but still risks getting unmaintainable if you try to reuse the same setup code for a wide range of tests.

·         Don’t unit-test configuration settings

By definition, your configuration settings aren’t part of any unit of code (that’s why you extracted the setting out of your unit’s code). Even if you could write a unit test that inspects your configuration, it merely forces you to specify the same configuration in an additional redundant location.

Conclusion:

Without doubt, unit testing can significantly increase the quality of your project. Many in our industry claim that any unit tests are better than none, but I disagree: a test suite can be a great asset, or it can be a great burden that contributes little. It depends on the quality of those tests, which seems to be determined by how well its developers have understood the goals and principles of unit testing.

MPEG2 PS/TS/Timecodes

MPEG-2 Program Stream Muxing

ffmpeg -genpts 1 -i ES_Video.m2v -i ES_Audio.mp2 -vcodec copy -acodec copy -f vob output.mpg

Note : In order to mux multiple audio tracks into the same file :
ffmpeg -genpts 1 -i ES_Video.m2v -i ES_Audio1.mp2 -i ES_Audio2.mp2 -vcodec copy -acodec copy -f vob output.mpg -newaudio

Note : In order to remux a PS file with multiple audio tracks :
ffmpeg -i input.mpg -vcodec copy -acodec copy -f vob output.mpg -acodec copy -newaudio

MPEG-2 Program Stream Demuxing

ffmpeg -i input.mpg -vcodec copy -f mpeg2video ES_Video.m2v -acodec copy -f mp2 ES_Audio.mp2

Note : This also works for files containing multiple audio tracks :
ffmpeg -i input.mpg -vcodec copy -f mpeg2video ES_Video.m2v -acodec copy -f mp2 ES_Audio1.mp2 -acodec copy -f mp2 ES_Audio2.mp2

MPEG-2 Start Timecode

ffmpeg -i <input_file> -timecode_frame_start <start_timecode> -vcodec mpeg2video -an output.m2v

Note : Start timecode is set as number of frames. For instance, if you want to start at 18:12:36:15, you will have to set -timecode_frame_start to 1638915 ( for 25 fps content ).

Comparison of encoder compression efficiency libdec265

Ref :https://github.com/strukturag/libde265/wiki/Encoder-comparison



Comparison of encoder compression efficiency

Cactus 1920x1080, 50 fps
Intra-frame distance set to 248 frames, x265 without WPP, HM in random-access configuration, otherwise default parameters. Date: 20/Aug/2014.

Johnny 1280x720, 600 frames

All encoders at default parameters (random-accesss), 21/Jul/2014.

Paris 352x288
Default parameters have been used for x265, x264, and f265. HM13 uses random-access configuration. Date: 18/Jul/2014.

Threading/Thread Pool X265

x265 creates a pool of worker threads and shares this thread pool with all encoders within the same process (it is process global, aka a singleton). The number of threads within the thread pool is determined by the encoder which first allocates the pool, which by definition is the first encoder created within each process.

--threads specifies the number of threads the encoder will try to allocate for its thread pool. If the thread pool was already allocated this parameter is ignored. By default x265 allocated one thread per (hyperthreaded) CPU core in your system.

Work distribution is job based. Idle worker threads ask their parent pool object for jobs to perform. When no jobs are available, idle worker threads block and consume no CPU cycles.

Objects which desire to distribute work to worker threads are known as job providers (and they derive from the JobProvider class). When job providers have work they enqueue themselves into the pool’s provider list (and dequeue themselves when they no longer have work). The thread pool has a method to poke awake a blocked idle thread, and job providers are recommended to call this method when they make new jobs available.

x265_cleanup() frees the process-global thread pool, allowing it to be reallocated if necessary, but only if no encoders are allocated at the time it is called.

Frame Threading

Frame threading is the act of encoding multiple frames at the same time. It is a challenge because each frame will generally use one or more of the previously encoded frames as motion references and those frames may still be in the process of being encoded themselves.

Previous encoders such as x264 worked around this problem by limiting the motion search region within these reference frames to just one macroblock row below the coincident row being encoded. Thus a frame could be encoded at the same time as its reference frames so long as it stayed one row behind the encode progress of its references (glossing over a few details).

x265 has the same frame threading mechanism, but we generally have much less frame parallelism to exploit than x264 because of the size of our CTU rows. For instance, with 1080p video x264 has 68 16x16 macroblock rows available each frame while x265 only has 17 64x64 CTU rows.

The second extenuating circumstance is the loop filters. The pixels used for motion reference must be processed by the loop filters and the loop filters cannot run until a full row has been encoded, and it must run a full row behind the encode process so that the pixels below the row being filtered are available. When you add up all the row lags each frame ends up being 3 CTU rows behind its reference frames (the equivalent of 12 macroblock rows for x264)

The third extenuating circumstance is that when a frame being encoded becomes blocked by a reference frame row being available, that frame’s wave-front becomes completely stalled and when the row becomes available again it can take quite some time for the wave to be restarted, if it ever does. This makes WPP many times less effective when frame parallelism is in use.

--merange can have a negative impact on frame parallelism. If the range is too large, more rows of CTU lag must be added to ensure those pixels are available in the reference frames. Similarly --sao-lcu-opt 0 will cause SAO to be performed over the entire picture at once (rather than being CTU based), which prevents any motion reference pixels from being available until the entire frame has been encoded, which prevents any real frame parallelism at all.

NoteEven though the merange is used to determine the amount of reference pixels that must be available in the reference frames, the actual motion search is not necessarily centered around the coincident block. The motion search is actually centered around the motion predictor, but the available pixel area (mvmin, mvmax) is determined by merange and the interpolation filter half-heights.

When frame threading is disabled, the entirety of all reference frames are always fully available (by definition) and thus the available pixel area is not restricted at all, and this can sometimes improve compression efficiency. Because of this, the output of encodes with frame parallelism disabled will not match the output of encodes with frame parallelism enabled; but when enabled the number of frame threads should have no effect on the output bitstream except when using ABR or VBV rate control or noise reduction.

When --nr is enabled, the outputs of each number of frame threads will be deterministic but none of them will match becaue each frame encoder maintains a cumulative noise reduction state.

VBV introduces non-determinism in the encoder, at this point in time, regardless of the amount of frame parallelism.

By default frame parallelism and WPP are enabled together. The number of frame threads used is auto-detected from the (hyperthreaded) CPU core count, but may be manually specified via --frame-threads

Each frame encoder runs in its own thread (allocated separately from the worker pool). This frame thread has some pre-processing responsibilities and some post-processing responsibilities for each frame, but it spends the bulk of its time managing the wave-front processing by making CTU rows available to the worker threads when their dependencies are resolved. The frame encoder threads spend nearly all of their time blocked in one of 4 possible locations:
blocked, waiting for a frame to process
blocked on a reference frame, waiting for a CTU row of reconstructed and loop-filtered reference pixels to become available
blocked waiting for wave-front completion
blocked waiting for the main thread to consume an encoded frame

Lookahead

The lookahead module of x265 (the lowres pre-encode which determines scene cuts and slice types) uses the thread pool to distribute the lowres cost analysis to worker threads. It follows the same wave-front pattern as the main encoder except it works in reverse-scan order.

The function slicetypeDecide() itself may also be performed by a worker thread if your system has enough CPU cores to make this a beneficial trade-off, else it runs within the context of the thread which calls the x265_encoder_encode().

Reference from: http://x265.readthedocs.org/en/latest/threading.html