1. Introduction

Analogue captures contain noise. The source of this noise is mainly due to the cables carrying the signal to your house. One should use smoothers to get rid of the noise without losing too much detail. The main difficulty is to find good a trade off between noise, the level of detail you want to keep in your clip and the encoding time. The best is to smooth before any resizing, because you will remove more noise this way.
There are two kind of smoothers:: temporal smoothers and spatial smoothers. Temporal smoothers work by looking at frames surrounding the current one and averaging corresponding pixels together if they look similar enough (and thus likely to be noise). Spatial smoothers work in a similar way, except they look at nearby pixels within the same frame. A few filters are a hybrid of the two, including NoMoSmooth, Convolution3D and PixieDust. This way you will remove more noise than when using only a spatial smoother. But remember if you use a temporal smoother with strong settings, you will see blending between frames. So, use low settings when using temporal smoothers.
Each approach has its advantages and disadvantages. The great thing about temporal smoothers is that they do a fantastic job getting rid of noise when you set them up just right, but they can also generate some very ugly artifacts, particularly when dealing with motion. Spatial smoothers, on the other hand, can be quite destructive to the details in your image, but are great with high-motion areas because those have few details and are moving too quickly to notice artifacts anyway. Ideally, one would like to apply a temporal smoother to relatively static areas and a spatial smoother to moving ones.
In this test several spatio-temporal smoothers are compared on an analogue capture. The clip can be downloaded here: [part1, part2] and it contains a performance of Dido (75 frames, captured with huffyuv). It has some slow and fast moving scenes, scenes with a uniform background and scences with some detail. Ivo (i4004) and I used the codec DivX3 and XviD for smoothing tests.

If this site is too slow, or the images don't load fast enough. They can be downloaded here.

2. Results

2.1 Behaviour of codecs when encoding clips with analogue noise

Most of us have read Doom9's famous codec comparisons (aimed at one CD rips with clean DVD sources). But how do the codecs perform on analogue captures which contains a lot of noise. In this subsection we will give some comments on this issue, considering the following codecs: DivX3, DivX5, FFVFW and XviD (no pre and post processing).

DIVX3.avi size: 436 KB (in this case k = 1024 bytes, therefore the used bitrate is 1200 kpbs / 1.024 = 1172 kbps)
DIVX5.avi size: 392 KB (DivX Pro 5.1.1, 1200 kpbs)
FFVFW.avi size: 390 KB (FFVFW 09-27-03, 1200 kpbs)
XVID.avi size: 384 KB (XviD v1.0 beta 3, 1200 kpbs)

The latter three encodings are undersized because of the small number of frames. For XviD and FFVFW this could be improved by highering the overflow settings. But since the desired size (440 KB) still couldn't be reached, we didn't bother doing this.

The codec settings used throughout the tests (unless stated otherwise):

DivX3 [Nandub]
I4-31, P2-9
Internal SCD-100%
1 pass

DivX5 [VdubMod 1.5.10.1]
no pre-processing
no b-frames
Standard Performance
2 passes

FFVFW [VdubMod 1.5.10.1]
I2-31, P2-9
MPEG quantization
no b-frames
[defaults on MotionEstimation tab]
2 passes

XviD [VdubMod 1.5.10.1]
I2-31, P2-9
MPEG quantization
no b-frames
Motion search precision: Ultra High
Trellis quantization
VHQ=4
2 passes

Standard script:

Avisource("D:\Test\huffyuv_raw2.avi")+Avisource("D:\Test\huffyuv_raw.avi")
KernelDeint(order=1, sharp=true)
BicubicResize(640, 480, 0, 0.75)
tweak(0.0, 0.8, 0.0, 1.0)

This script is used to encode using the above mentioned codecs. The results are given below.

There are several important issues where the behaviour of the codecs are different.

• Sharpness/bluriness. Of course this depends on your codec settings. For example, whether you use MPEG or H.263 quantization (the former is sharper than the latter), whether you use pre or post processing, whether you use VHQ (VHQ=0 is sharpest). etc. From most sharpest to bluriest with the used settings used: FFVFW/DivX3, XviD and DivX5. FFVFW and DivX3 on one hand is a bit sharper than XviD and DivX5 on the other hand.
  
  
• As explained, the difference in filesize is very large, since the clip is too short.
  

• When looking at the DivX5 and XviD clips, one can notice the following effect: when Dido's head turns to her left (frames 50-75), some parts of the background turn too (you can see that well if you turn on the brightness, or use zoomplayer to zoom in) Suspicious portion is in he left corner of video. This DivX5/XviD flaw is known for a while. Didee described it well in a recent doom9 thread called "Newest codecs: new artifacts?" (XviD forum):
  
  "3. "Tearing motion": Imagine you have a pretty dark scene. Something in the foreground is in focus, the background is almost flat colored (black sky, e.g.). Now, if the thing in the foreground starts moving, the background starts moving also (!), as if it was "teared" by the foreground object. This should be resolveable by a more sophisticated creation of motion vectors (don't create and trust predictors too blindly)."
  

Some screenshots from the XviD clip were taken, and a small slideshow was made to show the effect. Note, that it is much more clearly visible when watching the clip itself! On the right handside the tested codecs are compared (deinterlaced, resized, but no smoothing) with the source clip:

Tearing motion (source: XviD) Stop | Continue

Raw frame from the tested codecs (deinterlaced and resized) raw DivX3 DivX5 FFVFW XviD (trellis disabled, VHQ=0)

Does that mean that DivX5 and XviD shouldn't be used for encoding analogue captures? Well, yes and no. Yes, because there are good alternatives like DivX3 [Nandub] and FFVFW which don't have this undesirable effect. Yes, if you can't or don't want to denoise much. No, because this effect can be reduced using good denoisers. We will come back to this in the next subsection.

2.2 The comparison - settings

The following smoothers are considered (between brackets their settings):

The following scripts are used:

[1] PixieDust:

LoadPlugin("C:\Program Files\AviSynth2_temp\plugins\dustv5.dll")

Avisource("D:\Test\huffyuv_raw2.avi")+Avisource("D:\Test\huffyuv_raw.avi")
KernelDeint(order=1, sharp=true)
PixieDust(limit=5)
BicubicResize(640, 480, 0, 0.75)
Tweak(0.0, 0.8, 0.0, 1.0)

[2] Overlay with two different broadcastings:

clip1a=Avisource("D:\Test\huffyuv_raw2.avi")+Avisource("D:\Test\huffyuv_raw.avi")
clip2a=clip1a.KernelDeint(order=1, sharp=true)

clip1b=AviSource("D:\Captures\dido2.avi").Trim(608,632)+AviSource("D:\Captures\dido2.avi").Trim(1212,1262)
clip2b=clip1b.KernelDeint(order=1, sharp=true)

Overlay(clip2a, clip2b, mode="blend", opacity=0.5)
BicubicResize(640,480)
Tweak(0.0, 0.8, 0.0, 1.0)

[3] Combo's (see appendix for the qmf.avs script). For the low motion scenes PeachSmoother, TemporalCleaner or TemporalSoften are used, and for the medium/high motion scenes Mip or VagueDenoiser are used:

# QUANTIFIED MOTION FILTER v1.4
# EXAMPLE 2 : ADAPTIVE RESIZING FILTER

# LOADING QUANTIFIED MOTION FILTER SCRIPT
Import("D:\Test\Ivo\Wilbert\qmf.avs")

# LOW MOTION FILTER FUNCTION
# -> SHARP RESIZING + TEMPORAL ONLY
function Low_Motion_Filter(clip c)
{
ConvertToYUY2(c)
PeachSmoother(noiselevel=4.286, baseline=3.141, NoiseReduction = 60, Stability = 30, Spatial = 0)
ConvertToYV12()
BicubicResize(640, 480, 0, 0.75)
}

# MEDIUM MOTION FILTER FUNCTION
# -> NEUTRAL BICUBIC RESIZING + TEMPORAL & SPATIAL
function Medium_Motion_Filter(clip c)
{
c.MipSmooth(spatial=9, temporal=0, spatial_chroma=10, method="superstrong", downsizer="bilinear", upsizer="bilinear", scalefactor=0.60, weigh=true)
BicubicResize(640, 480, 0, 0.75)
}

# HIGH MOTION FILTER FUNCTION
# -> SOFT RESIZING + SPATIAL ONLY
function High_Motion_Filter(clip c)
{
c.MipSmooth(spatial=9, temporal=0, spatial_chroma=10, method="superstrong", downsizer="bilinear", upsizer="bilinear", scalefactor=0.60, weigh=true)
BicubicResize(640, 480, 0, 0.75)
}

# OPENING VIDEO SOURCE
Avisource("D:\Test\huffyuv_raw2.avi")+Avisource("D:\Test\huffyuv_raw.avi")
KernelDeint(order=1, sharp=true)
ConvertToYV12()

# APPLYING ADAPTATIVE RESIZING FILTER (USING QMF)
#QMF(debug=true)
QMF()

Tweak(0.0, 0.8, 0.0, 1.0)

2.3 The comparison - results

The denoisers are compared based on the following considerations: the amount of detail in the clip, uniformity of the background (for example, no artefacts in a uniform green background) and the absence of noise. The denoisers are listed from best to worst, based on our visual inspection. We will start by comparing the temporal denoisers: PeachSmoother, TemporalCleaner and TemporalSoften with Overlay and PixieDust. In this case only the low motion scenes are considered: The reason is that the temporal smoothers are applied on the low motion scenes, and spatial smoothers on medium/high motion scenes, using the filter Quantified Motion Filter. Which frames are detected as low motion or medium/high motion can be checked with QMF(debug=true).

frame 5 (XviD, 1200 kpbs): Overlay | Pixie | Peach | TemporalCleaner | TemporalSoften frame 21 (XviD, 1200 kpbs): Overlay | Pixie | Peach | TemporalCleaner | TemporalSoften frame 53 (XviD, 1200 kpbs): Overlay | Pixie | Peach | TemporalCleaner | TemporalSoften

The listing is as follows:

amount of detail:

1) PeachSmoother, TemporalCleaner, TemporalSoften
2) Overlay, PixieDust

Overlay and PixieDust are blurrier than PeachSmoother, TemporalCleaner and TemporalSoften.

uniform background:

I (Wilbert) always like that parts of a frame are uniform if they are supposed to be uniform. In this case it means that there are no artefacts in the green areas, and no artefacts on the skin of the girl.. 

1) Overlay
2) PeachSmoother, PixieDust
3) TemporalCleaner 
4) TemporalSoften

Overlay did the best job here, leaving no artefacts at all. PixieDust follows closely. In the first bunch of frames (say 25) PeachSmoother has more artefacts than TemporalCleaner and TemporalSoften in the green areas. But, Peach has a startup time. Hence, it takes a time before the filter starts to work. After the first 25 frames, PeachSmoother hardly leaves any artefacts. That's why it is listed above TemporalCleaner and TemporalSoften. Besides that, TemporalSoften gives sometimes noticable artefacts on the skin of the girl (see frame 39, 41, 43, 44, etc.).

absence of noise:

1) Overlay
2) PixieDust
3) PeachSmoother
4) TemporalCleaner, TemporalSoften

The ones with the least amount of detail, contains the least amount of noise. That's no surprise of course. However, PeachSmoother contains less noise than Temporalcleaner and TemporalSoften.

We will continue the comparison with the spatial smoothers: MipSmoother (note I used temporal=0, but I suspect it means that only temporal luma filtering is disabled) and VagueDenoiser. They will be compared to Layer and PixieDust. Only the medium / high motion scenes are considered.

frame 42 (XviD, 1200 kpbs): Overlay | Pixie | MipSmoother | VagueDenoiser frame 62 (XviD, 1200 kpbs): Overlay | Pixie | MipSmoother | VagueDenoiser

The listing is as follows:

amount of detail:

1) PixieDust, VagueDenoiser
2) Overlay
3) MipSmoother

PixieDust and VagueDenoiser has the most detail. Overlay is blurrier, and MipSmoother is the blurriest.

uniform background:

1) Overlay
2) PixieDust
3) MipSmoother
4) VagueDenoiser

Overlay did the best job here, leaving no artefacts at all. PixieDust follows closely. MipSmoother and VagueDenoiser have the most artefacts in the green background and on the skin of the girl VagueDenoiser more than MipSmoother. That's why VagueDenoiser is listed as last.

absence of noise:

1) MipSmoother, Overlay
2) PixieDust, VagueDenoiser

The ones with the least amount of detail, contains the least amount of noise. That's no surprise of course. VagueDenoiser has the most noise. PixieDust a bit less, and MipSmoother and Overlay are the cleanest.

We will conclude by making some general remarks about the pro's and cont's of the denoisers.

• Temporal denoisers will tend to ghost on medium / high motion scenes.
  
  
• PixieDust works quite well in that regard. It also ghosts (better to say, leaves some noise on motion) to some extent, but less than other temporal denoisers. Its motion estimation algorithm is therefore not perfect, and it tries to cover it with more of spatial processing (blurring).
  
  
• Overlay creates a too smooth image (because of excessive noise in the source, implying more pixels melted, and thus more blur).
  
  
• MipSmoother is a nice spatial filter, but not so good temporally.
  
  
• The winner among the temporal smoothers is without any doubts PeachSmoother. Which one of the spatial smoothers: VagueDenoiser or MipSmoother is the best depends on your taste, and also on which one is faster. MipSmoother is a bit blurrier than VagueDenoiser, but contains less noise (some people like this trade off, some people don't). Maybe they can be tweaked a bit to get acceptable results.
  
  
• In our mind the ultimate combo would be VagueDenoiser/MipSmoother (spatial part) and PeachSmoother (temporal part).
  
  
• Overlay denoises best (lest noise left), but it also loses most of the detail.
  
  
• PixieDust showed the strongest performance with regards to ghosting/detail preserved. It has least ghosting while preserving a reasonable amount of details.

The overall ratings:

1. PixieDust: winner - nice balance between non-ghosted and detailed image. The only downside is that it is slow as hell.
2. Overlay (blurred)
3. PeachSmoother + VagueDenoiser/MipSmoother (details are OK, ghosting isn't OK) (all other filters failed because of bigger ghosting problems)

2.4 Bitrates and Resolutions

A few words about bitrates and resolutions. The bitrates/resolutions we picked for this test are 512x384 at 800 kbps and 640x480 at 1200 kbps. They seem reasonable. Of course, you can't expect that 640x480 at 1200 kbps (movie) and 640x480 at 1200 kbps (sports, music videos) will look the same. In general, these sources need higher bitrates. With music videos and sports you can't really expect bigger benefits from 2 pass encoding modes, since there are no still scenes where the codec would be able to spare some bits.

3. Conclusions

Some conclusions and recommendations about the tested filters:

1. PeachSmoother + VagueDenoiser (or MipSmoother) is a nice combo where user will try to achieve best of both worlds; excellent denoising of still scenes of Peach, and smoothing out the ghosting artefacts on hi-motion levels. But it won't make wonders, Pixie will probably work slightly better on motion. Note that Peach's algorithm tries to leave elements that are important for sharpness (edges etc.) intact. You will notice that Peach leaves those vertical lines of vhs pretty noisy and shimmering.
   
   
2. Pixie has best quality on motion. We recommend this where Peach would normally ghost too much (higher motion/noise levels; in music videos etc), and you have a lot of time to do your encodings. The main issue is that Pixie is VERY slow (it's probably doing 2 passes internally), and it looks slightly blurrier than Peach.
   
   
3. Peach has best quality/speed ratio overall. We recommend this filter for long clips (like movies/tv-series), in case the combo's are too slow.
   
   
4. Overlay can be used for short (and difficult to denoise) clips if captures of different broadcastings (of the same clip) are available (like music clips).
   
   
5. Best filter is usually the slowest one (Murphy's Law).

Some conclusions about the encoders:

1. Project Mayo/Open Divx derived codecs act unpredictable in the event of noise+motion, artefacts range from hardly noticeable to very annoying (more noise means bigger troubles for XviD/DivX5). DivX3 [Nandub] and FFVFW don't have this issue.
   
   
2. Better denoising means less problems for XviD/DivX5. We doubt you can surpress the issue completely, no matter how well you cleaned the source.

4. Disclaimer

A small disclaimer, though it is actually more like a contest. It may very well be that you can find a set of filters which produce better quality than the filters we used. The source clips can be downloaded (see introduction), so everyone can take a try. If you have found something better, please contact us :)

5. Appendix

The qmf.avs script:

# QUANTIFIED MOTION FILTER (17/08/2003) by HomiE FR,
# a simplified version.

# MOTION ESTIMATION FUNCTION
function ME()
{
# SETTING MOTION LEVEL ACCORDING TO AVERAGE DIFFERENCE
global motion_level = (diff < threshold_lm) ? 0 : motion_level
global motion_level = (diff >= threshold_lm && diff <= threshold_hm) ? 1 : motion_level
global motion_level = (diff > threshold_hm) ? 2 : motion_level
}

# QUANTIFIED MOTION FILTER FUNCTION
function QMF(clip c, float "threshold_lm", float "threshold_hm", bool "debug")
{
# SETTING MOTION LEVELS THRESHOLDS
threshold_lm = default(threshold_lm, 4.5)
threshold_hm = default(threshold_hm, 12.0)
global threshold_lm = threshold_lm
global threshold_hm = threshold_hm

# ENABLING/DISABLING DEBUG INFORMATION
debug = default(debug,false)
global debug = debug

# SETTING PAST/PRESENT/FUTURE CLIPS
global clip = c

# GETTING OUTPUT RESOLUTION
width = Width(Low_Motion_Filter(clip))
height = Height(Low_Motion_Filter(clip))
global clip_resized = PointResize(clip,width,height)

# APPLYING MOTION FILTER ACCORDING TO MOTION LEVEL
c = ConditionalFilter(c,Low_Motion_Filter(clip),clip_resized,"motion_level","=","0")
c = ConditionalFilter(c,Medium_Motion_Filter(clip),c,"motion_level","=","1")
c = ConditionalFilter(c,High_Motion_Filter(clip),c,"motion_level","=","2")

# PRINTING DEBUG INFORMATION
c = (debug == true) ? ScriptClip(c,"Debug()") : c

# GETTING MOTION LEVEL THROUGH MOTION ESTIMATION
c = FrameEvaluate(c,"ME()")

# GETTING DIFFERENCES BETWEEN PAST/PRESENT/FUTURE FRAMES
c = FrameEvaluate(c,"global diff = 0.50*YDifferenceFromPrevious(clip) + 0.25*UDifferenceFromPrevious(clip) + 0.25*VDifferenceFromPrevious(clip)")

return c
}

# DEBUG INFORMATION FUNCTION
function Debug(clip c)
{
# PRINTING VERSION INFORMATION
c = Subtitle(c,"Quantified Motion Filter",x=20,y=30,font="lucida console",size=18,text_color=$FFFFFF)
c = Subtitle(c,"by HomiE FR (homie.fr@wanadoo.fr)",x=20,y=45,font="lucida console",size=14,text_color=$FFFFFF)

# PRINTING MOTION ESTIMATION INFORMATION
c = Subtitle(c,"motion estimation",x=20,y=85,font="lucida console",size=18,text_color=$FFFFFF)
c = Subtitle(c,"diff = "+string(diff),x=20,y=110,font="lucida console",size=16,text_color=$FFCCCC)

# PRINTING QUANTIFIED MOTION FILTER INFORMATION
c = Subtitle(c,"quantified motion filter",x=20,y=135,font="lucida console",size=18,text_color=$FFFFFF)
c = (motion_level == 0) ? Subtitle(c,"scene type = low motion",x=20,y=160,font="lucida console",size=16,text_color=$66FF66) : c
c = (motion_level == 1) ? Subtitle(c,"scene type = medium motion",x=20,y=160,font="lucida console",size=16,text_color=$66FF66) : c
c = (motion_level == 2) ? Subtitle(c,"scene type = high motion",x=20,y=160,font="lucida console",size=16,text_color=$66FF66) : c

return c
}

last edited on: 02/23/2004 | Authors: Wilbert & Ivo