An On-Line Machine Vision Flotation
Froth Analysis Platform
Charl P. Botha
October 1999
Thesis presented in partial fulfilment
of the requirements for the degree of
Master of Science
at the
University of Stellenbosch
Supervisor: Prof. D.M. Weber

Declaration
I, the under-signed, hereby declare that the work contained in this thesis is my own original work
and has not previously in its entirety or in part been submitted at any university for a degree.
Signature :
Date : April 2, 2004

Abstract
Froth flotation is a popular process used for the extraction of certain minerals from ore. The ef-
fective automatic control of this process has great financial importance. Utilising machine vision
as the measurement component of an automatic flotation control system has several advantages:
it is a cost effective solution due to its non-contacting sensors and consequent low maintenance,
as well as its low initial setup costs; it yields consistent and accurate parameters; and the mea-
surements it makes are available immediately, thus enabling real-time control and minimising the
effects of process disturbances rapidly and efficiently. In addition, machine vision can be used
as a research tool for investigating froth flotation, a process which is still not fully understood.
This work documents the development of an on-line machine vision flotation froth analysis plat-
form that is in use in the industry as the measurement component of an advanced flotation control
system and in the academic world as a powerful research tool. Two additional froth analysis ap-
plications, namely an embedded froth mobility sensor for use in the flotation industry and a
continuous froth video sampling and analysis research platform, are developed in the course of
this work.
An investigation into continuous video sampling and analysis of flotation froth is performed for
the first time and indicates that this mode of froth image acquisition yields more information
than traditional froth imaging methods. Recommendations are made with respect to the practical
application of continuous froth video sampling and analysis.

Opsomming
Skuim-flotasie is ’n gewilde proses vir die ontginning van sekere minerale uit erts. Die effektiewe
outomatiese beheer van flotasie dra groot finansie¨le impak. Daar is verskeie voordele verbonde
aan die gebruik van rekenaar-visie as die sensor-komponent van so ‘n outomatiese flotasie be-
heerstelsel. Dit is ’n koste-effektiewe oplossing vanwee die feit dat die sensors nie in aanraking
is met die flotasie nie en dus min onderhoud vereis. Boonop is die aanvanklike opstellingskoste
laag. ’n Masjien-visie stelsel lewer akkurate en betroubare metings en dit kan hierdie metings
baie vinnig beskikbaar maak wat intydse beheer van ‘n flotasie-aanleg vergemaklik. Daar kan
dan baie vinnig op proses-versteurings gereageer word. Masjien-visie is ook waardevol vir na-
vorsing oor flotasie, ’n proses wat nog nie heeltemal verstaan word nie.
’n Praktiese masjien-visie gebaseerde flotasie-skuim analise platform word ontwikkel. Hierdie
sagteware word gebruik in die flotasie-industrie as die sensor-komponent van ’n gevorderde
flotasie beheerstelsel asook as ’n kragtige navorsings-hulpmiddel in die akademiese weˆreld.
Twee bykomende toepassings word ontwikkel, naamlik ’n intydse flotasie-skuim mobiliteitsen-
sor vir gebruik in die flotasie-industrie en ’n navorsingsplatform vir die insameling en analise
van kontinue flotasie-skuim video-data.
Ondersoek word ingestel na die analise van kontinue flotasie-skuim video-data vir die eerste keer
in flotasie-skuim geskiedenis, en die resultate dui daarop dat hierdie metode van data-insameling
meer informasie oor die skuim inhou as tradisionele insamelings- en analise metodes. Daar
word aanbevelings gemaak omtrent die praktiese toepassing van kontinue flotasie-skuim video
monstering.

Acknowledgements
Dave Weber, my advisor for the past three years, has been a good friend and mentor and has
taught me everything that I know about research and coffee appreciation.
The DSP-Lab has been my research home for the past three years; the knowledge and experience
I have gained from interacting with its inhabitants are considerable. It has also been a brilliant
supplier of good strong coffee.
The creators of the free software that I depend on, such as GNU/Linux, CVS, LaTeX and my
all-time favourite editor, JED, deserve a lot of gratitude.
Marc van Olst and Francois du Plessis have been good friends and colleagues in my interaction
with Crusader Systems.
Gerhard “Goof” Esterhuizen and Mark Stoneman have taken the DSP-Group’s network to new
heights of efficiency and functionality and have made sure that I’ve always had machines and
good operating systems to work on.
All my friends make life a scream. Thanks guys.
My mother, Barry, Hermann and Etienne are very dear to me, and they’ve played a great part in
my education as human being.
Stella has redefined the term “supportive” during times when normal humans would have given
up. She is an amazing person, and my eternal love and gratitude belong to her.

Contents Contents
3.3 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.1 Linear Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.2 Circular Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.4 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.1 Histogram Accumulation . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4 Practical Implementation 40
4.1 Froth Flotation Plant Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Fluxar Flotation Control System . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.3 Implementations of Froth Surface Analysis . . . . . . . . . . . . . . . . . . . . 43
4.3.1 Analysis kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3.2 Fluxar-SP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3.3 Fluxar-Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3.4 Continuous sampling platform . . . . . . . . . . . . . . . . . . . . . . . 48
5 Performance and Impact 51
5.1 On Algorithm Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.1.1 Experiments with synthesised motion . . . . . . . . . . . . . . . . . . . 51
5.1.2 Qualitative judgement . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 Continuous Sampling Experiment . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2.1 Continuous Video Static Analysis . . . . . . . . . . . . . . . . . . . . . 54
5.2.2 A Useful Filter for Froth Parameters . . . . . . . . . . . . . . . . . . . . 57
5.2.3 Continuous Video Dynamic Analysis . . . . . . . . . . . . . . . . . . . 59
5.2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.3 Performance and Impact on Flotation Plants . . . . . . . . . . . . . . . . . . . . 65
vii

Contents Contents
6 Conclusions 67
6.1 Summary of Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
A Circular Statistics 74
A.1 Selected statistic measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
A.1.1 Mean Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
A.1.2 Median Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
A.1.3 Circular Variance and Standard Deviation . . . . . . . . . . . . . . . . . 75
A.2 Angular Distributions on the range (0,2pi/l) . . . . . . . . . . . . . . . . . . . . 77
viii

List of Figures
2.1 Image illustrating the over-segmentation that takes place if the marker extraction
and homotopic transform is not performed before the watershed segmentation.
The top image is the source image, the second is an un-preprocessed image that
has been run through the watershed, and the third image has been correctly pre-
processed and segmented. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Illustration of binary reconstruction. In the left image, the dark grey regions
represent the markers and the lighter grey regions the mask. Only connected
components of the mask which contain at least a pixel of a marker are kept and
the rest are eliminated. The result is shown in the right-hand image. . . . . . . . 13
2.3 Greyscale reconstruction. Analogously to binary reconstruction, the marker im-
age marks the parts of the mask that should be kept. . . . . . . . . . . . . . . . . 14
2.4 H-dome extraction. A constant h is subtracted from an image I to yield image
I −h. The image I is then reconstructed from I −h, and the result of this recon-
struction is subtracted from the original image thus yielding the h-dome image. . 15
2.5 Marker extraction. Note indicated position has two markers instead of one, and
this causes incorrect segmentation as can be seen in the last image in figure 2.1. . 15
2.6 Illustration of the homotopic modification. The marked minimum is enforced by
extending it downwards to the minimum possible value and non-marked minima
are filled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.7 Watershed terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.8 Sample watershed segmentation of flotation froth . . . . . . . . . . . . . . . . . 19
2.9 Oversegmentation due to extra markers . . . . . . . . . . . . . . . . . . . . . . 20
ix

List of Figures List of Figures
2.10 Results after oversegmentation correction to be compared with figure 2.1 (un-
corrected segmented image at bottom) and figure 2.1.1.1 (marker image pointing
out anomalous marker). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.11 Images illustrating working of marker bubble area ratio threshold fine froth de-
tection with MBRT parameter set to 0.004. . . . . . . . . . . . . . . . . . . . . . 23
2.12 Results of moving window SNE fine froth segmentation . . . . . . . . . . . . . . 24
2.13 Sample results of per-bubble motion analysis . . . . . . . . . . . . . . . . . . . 26
2.14 Output of the region-based motion estimation . . . . . . . . . . . . . . . . . . . 27
3.1 Froth image, image of top 2.5% of intensity values and histogram of froth image. 36
3.2 Example of area histogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1 Diagram of simple froth flotation circuit . . . . . . . . . . . . . . . . . . . . . . 41
4.2 Camera setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.3 Diagram of Fluxar Flotation Control System . . . . . . . . . . . . . . . . . . . . 43
4.4 Screen-shot of Fluxar-SP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.5 Screen-shot of sample Fluxar-SP image display window. From left to right: orig-
inal, bubble segmentation, region-based motion, bubble-based motion. . . . . . . 45
4.6 Screen-shot of Fluxar-SP database viewer with sample parameters . . . . . . . . 46
4.7 Screen-shot of Fluxar-SP Global Analysis Options dialogue . . . . . . . . . . . . 47
4.8 Screen-shot of Fluxar-SP Algorithm Options dialogue . . . . . . . . . . . . . . . 47
4.9 Photo showing FluxarSpeed prototype internal layout . . . . . . . . . . . . . . . 49
5.1 Small section of number of bubbles plot for 25 image per second sample set of
laboratory flotation cell froth illustrating number of bubbles sensitivity. . . . . . . 54
5.2 Segmented froth images corresponding to graph in figure 5.1. Images 144–149
left to right, top to bottom. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
x

List of Figures List of Figures
5.3 Section of average vertical pixel motion as calculated by region-based motion
calculation for 25 image per second sample set of laboratory froth. Compare
with the images in figure 5.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.4 Section of ellipticity figure plot for 25 image per second sample set of laboratory
froth. See figure 5.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.5 Plot of average bubble area/size over 400 images representing 16 seconds of on-
plant tank cell froth digitised at 25 images per second. . . . . . . . . . . . . . . . 58
5.6 Average bubble size signal filtered with α-trimmed mean filter with N = 10 and
α = 0.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.7 Input and segmented froth images corresponding to graph in figure 5.6. Images
117, 132 and 152 from top to bottom. . . . . . . . . . . . . . . . . . . . . . . . 60
5.8 Plot of average horizontal pixel motion as calculated by the region-based mo-
tion estimation over 400 images sampled at 25 images per second from on-plant
scavenger cell. The darker line is a version smoothed with an α-trimmed means
filter with α = 0.2 and N = 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.9 Plot of average horizontal pixel motion per image standard deviation as cal-
culated by the region-based motion estimation over 400 images sampled at 25
images per second from on-plant scavenger cell. The darker line is a version
smoothed with an α-trimmed means filter with α = 0.2 and N = 10. . . . . . . . 62
5.10 Section of plot of average vertical pixel motion for 400 images sampled at 25
images per second from laboratory flotation cell. . . . . . . . . . . . . . . . . . 63
5.11 Section of plot of average vertical pixel motion per image standard deviation
for 400 images sampled at 25 images per second from laboratory flotation cell.
Corresponds with figure 5.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.12 Images with motion vectors corresponding with positions 357, 360 and 363 in
graphs in figures 5.10 and 5.11. . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
A.1 Illustration of median direction . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
xi

List of Tables
1.1 Effect of process condition changes on bubble size . . . . . . . . . . . . . . . . 4
2.1 Average execution times for various algorithm groups on tank cell and scavenger
cell froth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.1 Extractable parameters and available post-processing . . . . . . . . . . . . . . . 30
3.2 Pre-processing of parameter arrays . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.1 Mean square pixel errors for horizontal, vertical and Euclidean motion calculated
over 100 test images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
xii

Keywords
image processing
machine vision
froth flotation
process control
sensor platform
greyscale morphology
motion analysis
mineral processing

Chapter 1
Introduction
1.1 Background and definitions
1.1.1 The Flotation Process
Froth flotation is a very widely used process for extracting certain minerals from ore. It is based
on the utilisation of differences in surface properties of particles of various minerals. Mineral-
rich ore is milled to a fineness that liberates most of the valuable particles and that will result in
particles that are small enough to stay attached to rising air-bubbles in the pulp [1]. This milled
ore is added to water. Reagents that modify the surface hydrophobicity of the particles are added
to the mixture and air is bubbled through the pulp. Hydrophobic particles attach to bubbles and
float to the surface where they form the froth layer and can be easily skimmed off, whereas the
other particles remain in the body of the pulp [2].
In direct froth flotation the valuable mineral particles are made to be hydrophobic so that they
attach to air bubbles when they collide with them in an agitated mixture of milled ore and water.
Other particles should ideally remain (or be made) hydrophilic so that they remain in the pulp.
Some of the reagents which are utilised in order to realise the flotation process are [3]:
frother Enables water to form a stable froth.
collector Adsorbed on mineral surfaces rendering them hydrophobic. Facilitates bubble attach-
ment.
activator Induces flotation in the presence of a collector which otherwise has no effect on the
target mineral.
1

Chapter 1. Introduction 1.2. Previous Work
Froth viscosity and mobility
Viscosity is influenced by the amount of hydrophobic particles present in the froth. The viscosity
and mobility are inter-dependent and play an important part in flotation performance [8].
Froth stability
When a froth is too stable, it could mean that the depressant flow is too low and that the bubbles
contain too much gangue (waste minerals). This obviously results in a low grade concentrate. If
a froth is unstable, its mineral-laden bubbles burst when they go over the cell-weir.
1.1.4.2 Froth types
Three primary froth types are identified by Moolman et al. in [7, 4]. Obviously the specific
appearances of the froth types differ for different flotation setups.
Ideal froth The bubbles are closely packed with bubble windows, and the froth is not too vis-
cous nor too runny. The bubbles are larger than those of the runny froth and smaller than
those of the sticky froth.
Runny froth The mineral load is low and the froth is watery, unstable and very mobile. The
bubbles are small.
Sticky froth The bubbles are large and can be elliptical. The froth is very stable and viscous.
1.2 Previous Work
In this section, we look primarily at work pertinent to the automatic control of the froth flotation
process utilising computational vision as the sensor component as well as just computational
vision froth surface analysis.
In 1992, Symonds and De Jager [10] first proposed the segmentation of images of the froth
structures into bubbles using morphological methods. These bubbles could then be counted and
analysed to characterise the segmented froth images. The morphological segmentation algorithm
that they proposed was a residual edge detector utilising a spherical structuring element (SE). By
opening and closing the image with a spherical SE (“rolling ball algorithm”) and then subtracting
the opened and closed images from the original image, a peak image and a valley image could be
5

Chapter 1. Introduction 1.2. Previous Work
obtained. These attempts, although promising, weren’t entirely successful due to the complexity
involved in trying to identify individual elements (bubbles) in the froth structure [8].
Moolman et al. took a different approach in [11] and used second order statistics to characterise
the textural properties of the surface froth. These textural properties were used as input features
to neural network classifiers. Using these techniques, they could accurately classify froth type
and detect froth mobility. Their method was refined in [8] to increase its classification accuracy.
The textural features that they extracted were numerical features on the neighbouring grey level
dependence matrix (NGLDM) and spatial grey level dependence matrix (SGLDM) based, both
second-order grey level statistics [12].
Eksteen’s work [1] utilised the image processing software and algorithms as developed by Mool-
man et. al, but focused more on providing a knowledge base for the research and development of
vision-based automatic froth control systems from a chemical engineering viewpoint. This work
concluded that a vision-based system for automatic flotation control was a commercially viable
venture and put forth the parameters for this viability. As a general overview of the factors perti-
nent to the development of a complete vision-based automatic flotation control system, this is a
valuable work. From an image-processing point of view, it contains a more detailed discussion
of the methods employed by Moolman et al.
Note that by this time, there were two distinct approaches in the froth surface visual analysis
business. One approach attempted to analyse the froth structure in a hierarchical and direct fash-
ion by attempting an accurate segmentation of the froth surface into bubbles, and then calculating
higher level parameters using this segmentation [10]. The other approach related distinct froth
types to textural properties and could indirectly derive other froth surface visual parameters from
texture.
The first successful attempt at full bubble segmentation was made in 1997 by N. Sadr-Kazemi
and J.J. Cilliers of UMIST [13]. In this research, the morphological watershed algorithm was
used to segment froth images into bubbles. It is also stated that “there have been few attempts at
estimating the complete bubble size distribution and relating that to the flotation performance”.
The dynamic (motion) analysis of flotation froth is also significant as it is interdependent with the
froth viscosity. Motion blur caused by slightly longer than normal exposures on video capturing
equipment was used by Moolman et al. [9] and Eksteen [1] to calculate froth mobility. They
found that the NGLDM second moment as calculated on one of these blurred images gave an
accurate measure of froth speed. This conforms with their paradigm of classifying froth types
according to surface texture. Other work on calculating froth mobility [14] used standard block
matching algorithm (BMA) based methods and compared these with bubble-based motion. BMA
based methods calculate motion by attempting to match small regions of a source image with
corresponding regions on a target image, where the source and target image are images of the
same scene taken temporally apart. The bubble-based motion estimation was found to be more
suitable, contrary to our results.
6

Chapter 1. Introduction 1.3. Overview of this work
A complete vision-based expert system for flotation froth control is documented in [15]. Bubble-
segmentation is performed, but the description of the method used is quite vague, although it
does seem related to the watershed. This system, although commercially marketed, does not
seem very successful.
JKFrothCam is a commercially available system for the on-line monitoring of flotation froth that
was developed at the Julius Kruttschnitt Mineral Research Centre (JKMRC) in Australia [16]. It
captures, digitises and analyses video from cameras monitoring the flotation froth surface. It is
able to calculate three surface characteristics, namely average bubble size, froth texture and froth
velocity.
For the froth texture, a texture unit number is generated at each point by performing a certain
calculation (measuring the difference between neighbouring pixels) within a 3× 3 pixel matrix
at that pixel. A histogram, or texture spectrum, of number of occurrences vs. texture unit number
is then generated. The height of the middle peak in this spectrum is used to classify bubble size,
whilst the whole spectrum is compared to pre-calibrated representative froth spectra to determine
the froth type.
The velocity is calculated using what the JKMRC calls “pixel tracing”. Judging by available
documentation this is a form of the BMA for motion estimation.
JKFrothCam is able to estimate certain metallurgical properties of the froth, such as ash con-
centration, using a simple linear model with the three extracted parameters (type, bubble size,
velocity) as variables. The model parameters are regularly calibrated using a self-tuning Kalman
filter.
1.3 Overview of this work
This work documents the development, implementation and investigation of a complete set of
flotation froth surface computational vision analysis algorithms. These algorithms are able to
perform a complete static and dynamic analysis on digital images of the froth surface.
The static analysis includes a full bubble segmentation where the perimeter of each bubble is
determined using watershed morphology. In addition to the number of bubbles, parameters like
per-bubble area, perimeter length, orientation angle and ellipticity are calculated from the seg-
mentation results. As part of further post-processing, histograms and statistic values are calcu-
lated, taking special care with angular data.
Textural parameters are also calculated as part of the static analysis and can be used as an auxil-
iary indication of parameters like bubble-size. In addition, areas that contain fine froth, i.e. froth
that is too fine to segment into distinct bubbles, are tagged as such. This is a very important step,
7

Chapter 1. Introduction 1.5. Thesis Organisation
In order to facilitate the development of this package, a practically re-usable froth analysis code
kernel was created that can be re-deployed in any number of practical applications. One such
application is the embedded froth surface mobility analyser that was developed in the course of
this work.
In contrast to the long analysis times required by the bubble segmentation implementations men-
tioned in section 1.2, this work is able to perform full analyses in near-realtime, enabling it to be
deployed on flotation cells in a closed-loop control system.
The sensor platform has been successfully integrated with a practical flotation plant control sys-
tem, in which capacity it is performing very satisfactorily. It is one of only two such systems
known in the world that are currently being marketed, and it far outstrips the competing sensor
platform with respects to functionality.
No known prior work has investigated froth surface analysis on continuous full-rate video data.
We have performed experiments in this regard and our results are documented in this work. A
re-usable application that is able to gather large amounts of froth images at continuous rates and
analyse these off-line was developed using the froth analysis kernel. This is a valuable research
tool.
This work adds to the small formalised knowledge-base available to practitioners and researchers
in the machine vision analysis of flotation froth.
1.5 Thesis Organisation
Chapter 2 documents the specific image processing algorithms and principles that were used
to analyse the froth structure images. At top level, the chapter can be divided into static and
dynamic analyses.
In chapter 3 we detail how the information yielded by the image processing algorithms and
techniques is processed and converted to usable parameters.
A description of the implementation of the whole froth surface visual parameter extraction sys-
tem is given in chapter 4. This also describes the complete control platform of which the sensor
platform is an important part.
Chapter 5 presents the results obtained from both laboratory and plant operation.
Chapter 6 contains a summary of this work, the contribution therein and re-iterates the important
knowledge that was gained. Possibilities for future work on this subject are also discussed.
9

Chapter 2. Image Processing 2.1. Static Analysis
Figure 2.1: Image illustrating the over-segmentation that takes place if the marker extraction and homotopic trans-
form is not performed before the watershed segmentation. The top image is the source image, the
second is an un-preprocessed image that has been run through the watershed, and the third image has
been correctly pre-processed and segmented.
12

Chapter 2. Image Processing 2.1. Static Analysis
enforced minimum
non-marked
minima filled
marker
Figure 2.6: Illustration of the homotopic modification. The marked minimum is enforced by extending it down-
wards to the minimum possible value and non-marked minima are filled.
down into that specific local minimum, is the catchment basin of the local minimum. The borders
that separate catchment basins belonging to different local minima are called watershed lines or
just watersheds. Figure 2.7 illustrates the terms just defined.
The purpose of the greyscale watershed is to extract watershed lines, i.e. contiguous lines of
high-intensity pixels separating local intensity minima in the image. An alternative approach to
understanding the watershed that is closer to the implementation that we used is the “immersion”
definition [17]. If we were to puncture the landscape at all its minima and then immerse it in a
lake, the water would progressively fill all the catchment basins. At each pixel where the water
from two different minima would merge, we build a dam wall. When the whole landscape has
been fully immersed, the set of resultant dam walls represent the watershed lines that delineate
the whole landscape into its constituent catchment basins.
The immersion definition also makes the applicability of the watershed algorithm to bubble seg-
mentation obvious. Recall that we need to segment out each and every bubble in an image of
flotation froth surface. Each bubble is structurally an irregular dome, with the lowest elevation
or altitude being where bubbles meet, i.e. the domes intersect, and the highest point usually co-
inciding with the localised light reflection if the light source is close to the the normal axis of the
froth surface. If we approximate the elevation at any point with the recorded pixel intensity at
that point, which is intuitively and empirically a valid approximation, we have a digital elevation
map of the froth surface. If we invert the elevation map, i.e. turn the maxima into minima and
vice versa, and perform the watershed immersion on the inverted elevation map, the result is a
set of watershed lines coinciding with bubble boundaries; in other words, a segmentation of the
bubbles in the froth image. In practice, the result is a tessellation of the image into catchment
17

Chapter 2. Image Processing 2.1. Static Analysis
local minimum a
local minimum b
watershed
watershed
watershed
catchment
basin b
catchment
basin a
Figure 2.7: Watershed terminology
basins where each pixel is tagged depending on the catchment basin that it belongs to.
With this explanation in mind, one can also see why the homotopic modification is so important.
After inversion, bubbles, being irregular domes, have numerous spurious local minima. From the
immersion definition, one can see that extra watershed lines will result for all these extra local
minima. The homotopic modification fills these extra local minima so that the watershed extracts
only the desired bubble boundaries.
Our implementation is the highly optimised watershed algorithm from [17]. It executes the
watershed by performing immersion in a very computationally efficient method. The reader is
referred to the reference for details. Previous to this specific implementation of the watershed, it
was not possible to perform such a segmentation in near-realtime.
Figure 2.8 shows a sample result of the watershed on a froth image that was homotopically
modified beforehand. Note how well the watershed lines (indicated in black) follow the bubble
boundaries.
2.1.1.4 Post-segmentation Over-segmentation Correction
As we have just seen, the homotopic modification uses a marker image to ensure that the wa-
tershed segments out only one catchment basin per bubble. This process assumes that a single
marker will be extracted for each bubble. This assumption almost always holds true. There are
occasions however when the irregularity of the bubble domes causes major discontinuities, e.g.
darker lines, in the localised light reflections that are used as markers. See figure 2.5 for an ex-
ample. These discontinuities result in more than one marker being found for that specific bubble,
which in turn results in that particular bubble being over-segmented, i.e. segmented incorrectly
18

Chapter 2. Image Processing 2.1. Static Analysis
Figure 2.10: Results after oversegmentation correction to be compared with figure 2.1 (uncorrected segmented
image at bottom) and figure 2.1.1.1 (marker image pointing out anomalous marker).
2.1.2 Other textural properties
As noted in section 1.2, Moolman et al. based their froth classification efforts on the extraction
of SGLD and NGLD matrices and the calculation of features on these matrices. Some of this
functionality has been included in this work for reasons which will be discussed shortly.
The Neighbouring Grey Level Dependence Matrix (NGLDM) [12] as calculated on an image or
image region contains information about the relationship between each of the pixels and all its
neighbouring pixels at a certain distance d. The matrix is represented as a two-dimensional array
Q of size Ng rows by Nr columns, where Ng is the number of possible grey levels and Nr is the
number of possible neighbours to a pixel in an image. The Q matrix is computed for a specific
positive integer value of distance d and difference a. For every pixel, the number of times that
its grey value and a neighbouring pixel’s grey value differ by a or less are counted. The value at
row i and column j in the Q matrix is incremented, where i is the grey value of the pixel under
examination, and j is the value itself.
This matrix relates to the texture information in an image or image region. It is invariant to image
rotation as well as to linear grey level transformations or shifts of the grey level spectrum. For
example, if the brightness of an image were to be increased without resulting in any values being
truncated, the NGLDM would look exactly the same as in the darker case.
Scalar image features can be calculated on the Q matrix, such as the Small Number Emphasis
(SNE), Large Number Emphasis (LNE), Number Non-uniformity (NNU), Second Moment (SM)
and Entropy (ENT). Each of these scalars indicates some qualitative measure. The SNE is a
measure of the fineness of the image, the LNE is a a measure of the coarseness of the image,
the SM is related to the homogeneity of the image. The NNU and ENT are also related to the
21

Chapter 2. Image Processing 2.2. Dynamic Analysis
Figure 2.13: Sample results of per-bubble motion analysis
the corresponding block in the target image. Again, the source image and target images are of
the same scene but taken a short time apart. The matching target block is found by calculating
some similarity measure between the source block and several blocks in a target image search
region. Of these candidate blocks, the one that achieves the highest similarity measure is selected
as the matching block. The difference in position between the source block and the target block
represents the motion that was undergone by this block in the time interval between images.
For this application, the Maximum Cross-Correlation (MCC) matching criterion was used. Each
source block is cross-correlated with all the possible overlapping target blocks in its search region
in the target image, and the position at which maximum cross correlation is achieved is selected
as the position of the matching target block. The BMA with MCC matching criterion is a form of
feature-detection by matched filtering [22, 23] where the matched filter in this case is the source
block itself. The motion vector V i(m,n) measured from image Ii to a following image Ii+1 for a
block centered at pixel location (m,n) is determined by:
V i(m,n) = (xd −m,yd −n)
where
(xd,yd) = arg max
x,y∈Ω
(
∑M/2p=−M/2 ∑
M/2
q=−N/2 Ii(m+p,n+q)Ii+1(x+p,y+q)
√
∑M/2p=−M/2 ∑
M/2
q=−N/2 I
2
i (m+p,n+q)∑M/2p=−M/2 ∑
M/2
q=−N/2 I
2
i+1(x+p,y+q)
)
The motion-block is M by N pixels and the search region Ω is a block congruent with the motion-
block but larger. The normalisation is very important. It eliminates the effect of differing signal
energies between the search block and the block in the search area that it’s being matched with.
26

Chapter 3. Parameter Extraction 3.2. Extractable parameters
Parameter Statistics Circular Statistics Histogramming Once-off
area/size X X
blue X X
bubble-count X
ellipticity X X
fine froth ratio X
green X X
intensity X X
orientation X X
perimeter X X
red X X
SNE X
valid froth ratio X
velocity (bubble) X X X
velocity (area) X X X
Table 3.1: Extractable parameters and available post-processing
3.2 Extractable parameters
Several parameters describing the physical properties of the froth surface can be calculated from
the low-level image processing data. Most of these parameters are large vectors, describing e.g.
either all the bubbles in an image or describing motion vectors in a region of interest, and will
thus be further processed at a later stage by a combination of statistic calculation, circular statistic
calculation and histogramming.
3.2.1 Area/Size
Operator-perceived bubble size is one of the most important parameters describing the froth
surface. We approximate operator-perceived bubble-size with the bubble-area, i.e. the number of
area units, or pixels, that the segmented two-dimensional projection of the bubble occupies. This
is a safe assumption as absolute bubble-size is not required, but rather relative bubble-size. That
said, the area of the two dimensional projection is very closely related to the true surface area of
a bubble, with the estimation error relative to the true bubble size, the degree of curvature (more
specifically the specific shape of the bubble-dome) and the height of the bubble if the projection
takes place along the normal axis.
Bubble area is calculated by counting the exact number of pixels contained in its tessellated two-
dimensional projection. This is done for each and every bubble in the image. The counting is
done by performing a sum on the run length pixel information for that bubble. Converting the
30

Chapter 3. Parameter Extraction 3.2. Extractable parameters
Although it is calculated and available for use, no work was done on investigating or utilising its
relationship with froth characteristics.
3.2.5 Ellipticity
For a two-dimensional shape S containing N binary-valued pixels, the centre of mass (m,n) is
defined as [22]:
m =
1
N ∑(m,n)∑∈S m, n =
1
N ∑(m,n)∑∈S n
and the (p,q) order central moments are:
µp,q = ∑
(m,n)
∑
∈S
(m−m)p(n−n)q
Utilising the run length encoded pixel data the centroids and the second order central moments,
µ1,1, µ2,0 and µ0,2, are calculated. The covariance matrix for the shape (where each pixel is
considered as a two-dimensional feature) can then be constructed as follows [25, p 138]:
[
σ11 σ12
σ21 σ22
]
=
[
µ20 µ11
µ11 µ02
]
Where σ11 and σ22 are the variances of the horizontal and vertical dimensions, and σ12 = σ21
is the covariance (measurement of joint relationship) between the horizontal and vertical dimen-
sions.
The best-fit ellipse for our two-dimensional shape, i.e. the binary blob-image of a bubble, is an
ellipse with a second moment (covariance matrix) equal to that of the shape [22, p 394]. This is
akin to approximating the probability distribution of the two-dimensional features, or pixels, with
a bivariate normal distribution, in which case contours of constant density are ellipsoids centered
at the centre of mass (m,n) with axes defined by ±c
√
λiei, where c is a constant defined by the
constant density chosen, λi is the i’th ordered eigenvalue and ei is its corresponding normalised
eigenvector of the covariance matrix [26, pp 335–337].
We define the ellipticity ε of this two-dimensional ellipsoid to be the ratio of the longest axis to
the shortest axis, i.e.
ε =
|± c
√
λaea|
|± c
√
λbeb|
=
λa
λb
In other words, eigenvalues are calculated for the covariance matrix of the segmented bubble-
shape under examination, and the ratio of the largest eigenvalue to the smallest eigenvalue is
calculated, as it represents the ellipticity of the best-fit ellipsoid.
32

Chapter 3. Parameter Extraction 3.2. Extractable parameters
This parameter is related to the viscosity and stability of the froth. An ellipticity of unity repre-
sents a circular bubble whilst a larger ellipticity represents a more elliptical or elongated bubble,
which usually appears in a viscous stable froth [8].
3.2.6 Fine froth/Valid Froth Ratio
The total area of the regions classified as fine froth froth by the fine froth detection algorithm
in section 2.1.3.1 is calculated as a ratio of the total image area. This is the fine froth ratio and
can be used as a direct measure of the amount of fine froth in an image and consequently as an
indication of bubble-size, especially when the ratio is large. When the ratio is relatively small,
bubble-size information gained from segmentation is more significant.
The valid froth ratio is the ratio of valid froth area to total image area, where valid froth area is
simply the difference in area between total area and fine froth area.
3.2.7 Orientation
The orientation is defined as the clockwise angle between the longest axis of the best-fit ellipsoid
(see 3.2.5) and the horizontal. As discussed, the ellipsoid has axes ±c√λiei, with ei being the
i’th eigenvector of the shape’s covariance matrix. The vector em corresponding to the largest
eigenvalue is used to calculate the angle, where the angle θ is defined as:
θ = tan−1
(
em[1]
em[0]
)
The angle is adjusted to be on [0◦,180◦].
The orientation can be used to determine to which degree ellipsoid bubbles are elongated in
the direction of froth motion. The more the elongation is in the direction of motion, the more
significantly it indicates viscosity.
3.2.8 Perimeter
The perimeter is the pixel perimeter of the segmented bubble-projection. A boundary tracing
algorithm from [27] is performed on the bubble-segmented tessellation (all pixels belonging to
a certain bubble are tagged uniquely, i.e. have a specific colour) of the froth surface image. The
boundary tracing algorithm yields a list of pixel chain codes describing the Euclidean perimeter
of the bubble.
33

Chapter 3. Parameter Extraction 3.3. Statistics
Each chain code indicates the direction in which the next pixel can be found. There are eight
directions, of which four are diagonal and four are vertical or horizontal. By traversing the list
of chain codes and adding one pixel to the perimeter total for every horizontal or vertical chain
code, and
√
2 pixels for every diagonal chain code, the Euclidean perimeter of the bubble can be
calculated.
The bubble-perimeter is very closely related to the bubble-size.
3.2.9 Region-based Motion
The region-based motion estimation documented in section 2.2.2 results in a matrix of regu-
larly spaced motion vectors all within the specified region of interest. The regular spacing is
determined by the block size, as each vector denotes the motion that a specific partitioned block
underwent from one image to the next.
These motion vectors are initially available as differential horizontal and vertical motion pairs,
but polar representations are calculated as well. In contrast to the bubble-based motion estima-
tion, summarising information (such as means, histograms, etc.) calculated from these vectors
are more readily interpretable as they are regularly spaced.
Our parameter choices are documented in sections 2.2.2 and 5.2.
3.2.10 SNE
The SNE was discussed in section 2.1.2. This parameter is calculated once for a whole image.
No post-processing is performed on it.
3.3 Statistics
All the multi-valued extracted parameters, i.e. parameters which have distinct values for each
bubble, for each pixel or for each partitioned motion block in an image, are summarised with
statistic values that are useful in online flotation control systems. These values are: minimum,
maximum, standard deviation, mean and median. The mean refers to the arithmetic mean of the
data and the median to an actual data value that has half the data values greater than it and the
other half smaller than it. The standard-deviation is a measure of the spread of the data about its
mean.
Before these values are calculated on the extracted data, parts of the data which might affect
34

Chapter 3. Parameter Extraction 3.3. Statistics
the accuracy of the calculated statistics are discarded. Recall that all regions that are classified
as fine froth are ignored in the calculation of any statistics or histograms. Furthermore, for the
bubble area and perimeter parameters, the highest 0.5% and lowest 5% of parameter magnitudes
are discarded when calculating statistic values or histograms. This is to eliminate any possible
outlier data. Although the convention is to discard 5% from both the higher and lower ends, it
was empirically found that 5% from the higher values discards important samples individuals
when working with bubble area and perimeter. This is a form of the L-estimator defined in [28]
where the L-estimate ˆθ is defined as:
ˆθ =
N
∑
i=1
aiXi
with Xi being the sorted series of values to be estimated and ai a corresponding set of coefficients.
We define a number of the first and last ai coefficients to be zero, and the remaining ones to be
1
N−d where d is the total number of ai coefficients that are set to zero. In this way, we eliminate
outliers before calculating our estimates.
With smaller bubbles where we find that less than 10 pixels can constitute a distinct segmented
bubble, the best-fit ellipsoid that is used to calculate bubble orientation and ellipticity tends
toward circularity. In most cases, this is an inaccurate approximation that takes place because of
spatial under-sampling, and we do not want these results to bias our statistics or histograms. For
that reason, we discard the orientations and ellipticities belonging to the 30% smallest bubbles.
In addition, 0.5% of the largest bubble ellipticities are discarded to account for outliers.
With colour and intensity data, it was found that eliminating 2.5% from the greatest and the
smallest magnitude data points compensated for the localised reflections found on the bubbles
and for the darkest outliers. Intensity and colour information should not be affected by the
localised reflections, as these are artifacts of the light sources. Figure 3.1 shows a froth image
and then an image consisting of just the top 2.5% of its intensity values. Note that these include
many of the brightest structures, but that 2.5% is a very modest estimate. Notice in the histogram
the clustering at the highest and lowest values, representing the values that we discard.
None of the other parameter sets are culled in any way. Discarding angular data such as bubble
orientation is not easily justifiable, as the definition of an outlier is absolutely dependent on where
the reference diameter is chosen. Table 3.2 summarises the pre-processing that takes place.
The calculation of these statistic values is different for linear data (such as bubble area) and for
angular data (such as bubble orientation). Table 3.1 shows which methods were used for which
parameters. A mark in the “Statistics” column indicates that normal linear statistics are calcu-
lated, and a mark in the “Circular Statistics” column indicates that special circular calculations
were performed. The next two sections outline the specific calculations that were used.
35

Chapter 3. Parameter Extraction 3.4. Histograms
3.4.1 Histogram Accumulation
Histograms can be calculated for each image that is captured and analysed. These analyses
usually take place at regular intervals, and with each new analysis, the histogram of the previous
analysis is replaced with a new one. Instead of this behaviour, histograms can be accumulated
over time to yield an accumulative histogram which represents the nature of the froth over a
certain time interval. In this way, the accumulated histogram is able to contain information about
past perturbations in a flexible way by acting as an estimator and in fact smoothing histograms.
The accumulation is performed with an exponential forget factor. Let hn and hn be the accu-
mulative and instantaneous histograms at discrete time n and λ an adjustable forget-factor that
satisfies |λ|<= 1, then:
hn =
n
∑
k=0
λkhn−k
This can be implemented as a recursive system as follows:
hn = hn +λhn−1
Note that a forget-factor λ = 1 means that no forgetting takes place, i.e. all past histograms
contribute equally to the current accumulative. The smaller the forget-factor, the more the accu-
mulation discounts the effect of past histograms, up to the limiting case where λ = 0 and only
the current histogram makes a contribution.
Such an accumulative histogram can be used to examine the non-instantaneous nature of the
froth. The forget-factor ensures that effects of process disturbances can be gradually discounted
in the accumulative histogram. When the accumulated histogram is examined, it is useful to
scale it as follows:
1
1+λ+λ2 +λ3 . . .+λn hn =
1−λ
1−λn+1 hn
This yields a histogram which represents the accumulative nature of the froth, but whose values
are in range for a normal instantaneous histogram.
39

Chapter 4. Practical Implementation 4.2. Fluxar Flotation Control System
Figure 4.2: Camera setup
42

Chapter 4. Practical Implementation 4.3. Implementations of Froth Surface Analysis
camera
camera
camera
camera
Fluxar-SP
Online ModelGen
SQL
SQL
InTouch SCADA
PLC Advisor
Computer 1
Computer 2
Figure 4.3: Diagram of Fluxar Flotation Control System
quality cameras, cabling and frame grabber cards can not be overstated. The price of these items
is small in comparison to the gains brought about by increased parameter accuracy. All possible
steps that can improve the lighting conditions and camera stability should be taken. The image
analysis algorithms are relatively robust, but improved input equates to more accurate output.
PAL video cables run from the cameras to the control room where the computer hardware is situ-
ated. These video signals are sampled by Fluxar-SP, the full-featured froth analysis implementa-
tion discussed in section 4.3, which performs any combination of configurable froth analyses on
this video input. At the moment, dynamic analysis is performed on the even and odd fields of a
single image. These fields are exposed 20ms apart, as per the PAL standard. It stores all the froth
information that it extracts into an SQL database, from whence it is accessed by Online Model-
gen, an application which performs filtering, basic mathematical manipulation and fuzzy model
decision making on this information. This output is stored in another SQL database, where it is
accessed by Intouch SCADA (Simultaneous Control And Data Acquisition) and used to perform
automatic control via the plant’s PLCs and/or to advise the plant’s operators about recommended
control actions. Figure 4.3 shows a block diagram of the Fluxar Control System. The reader is
referred to the Crusader [29] and Fluxar [30] web sites for further information.
4.3 Implementations of Froth Surface Analysis
The image analysis of chapter 2 and the parameter extraction of chapter 3 were implemented in
several distinct applications with differing functionality and applicability. Most of the analysis
functionality was abstracted in a shared analysis kernel, which will be discussed first.
43

Chapter 4. Practical Implementation 4.3. Implementations of Froth Surface Analysis
Figure 4.4: Screen-shot of Fluxar-SP
Figure 4.5: Screen-shot of sample Fluxar-SP image display window. From left to right: original, bubble segmenta-
tion, region-based motion, bubble-based motion.
45

Chapter 4. Practical Implementation 4.3. Implementations of Froth Surface Analysis
Figure 4.6: Screen-shot of Fluxar-SP database viewer with sample parameters
deactivated. Note that these parameters can be different for each acquisition channel. Figure 4.8
shows the algorithm options dialogue, where the user may configure combinations of parameters
and/or histograms to be calculated for each acquisition channel.
It is able to acquire images from up to four multiplexed video-channels using either a Matrox
Meteor or IntegralTech FlashPoint 3D Plus frame grabber card. As an added feature it is able to
acquire from bitmap images stored on disc. As the on-line acquisition methods can be configured
to store images to disc as they capture, these images can later be analysed off-line, in order to
diagnose process disturbances after the fact. Note that in addition to being a crucial part of the
automatic flotation control system, Fluxar-SP is a valuable research tool.
The Fluxar-SP front-end was developed using Borland Delphi 4.0 and this instance of the analysis
kernel built with Borland C++ Builder 3.0. Development started on an initial third-party attempt
at a flotation froth sensor-platform. More than 75% of the functionality was developed as part of
this work.
Fluxar-SP is used actively in research environments in South Africa and Australia and has been
efficiently functioning as the sensor component of a flotation control system in advisory mode
on a real platinum extraction plant in South Africa since the end of 1998, and as the sensor
component of a closed-loop flotation control system on this plant since the middle of 1999.
46

Chapter 4. Practical Implementation 4.3. Implementations of Froth Surface Analysis
Figure 4.7: Screen-shot of Fluxar-SP Global Analysis Options dialogue
Figure 4.8: Screen-shot of Fluxar-SP Algorithm Options dialogue
47

Chapter 4. Practical Implementation 4.3. Implementations of Froth Surface Analysis
4.3.3 Fluxar-Speed
Froth stability and mobility are very important measurements in automatic flotation control [9].
This importance was realised in another application of the froth analysis kernel. FluxarSpeed is
an embedded application that is able to perform region-based motion analysis in real-time and
output average motion as an analogue signal, as values from its serial port and as a continuous
reading on its mounted current meter. Depending on the configured motion analysis parameters,
the prototype system can analyse up to 15 images per second.
It was designed for installation in flotation plants which do not require the extended functionality
of Fluxar-SP. A flotation plant would mount several of these units at different positions and use
the analogue outputs as measurement signals in their existing control platforms. By keeping
froth mobility at predefined “good” levels, extraction plants can be run at greater efficiency.
FluxarSpeed consists of x86 architecture hardware running the Linux operating system. Figure
4.9 shows the internal layout of the FluxarSpeed prototype. At bottom left the Intel Pentium III
450MHz CPU can be seen, and at top left the built-in greyscale camera and PCI grabber card.
When installed, the system can be configured via its built-in network interface by using telnet, or
via one of the serial ports, by connecting a portable computer with a serial terminal application.
All the parameters of the area-based motion analysis can be configured in this way. The software
on the product can also be updated using these methods. The system is designed for full-time
low-maintenance operation, whilst maintaining a very high-level of in-place configurability and
upgrade-possibilities. This makes for a low-cost high-gain measurement device.
Outokumpu Mineral Processing Oy of Finland will be marketing the Fluxar-Speed system on an
international basis. For this purpose, the system is being ported to an industrial embedded PC
platform. The complete software aspect of FluxarSpeed was engineered as part of this work.
4.3.4 Continuous sampling platform
The third implementation of the froth analysis kernel was developed purely for research purposes.
It consists of a continuous sampling component and a batch analysis component. The continuous
sampling component is able to capture video sequences as continuous sets of distinct images and
store them on disc. The batch analysis component is able to perform the full froth static and
dynamic analysis on these stored image sequences in an off-line fashion and store the extracted
parameters on disc.
Nowhere in documentation has this type of continuous froth video analysis been performed. It
is especially important in examining parameter trends which occur at a higher frequency than
the customary sampling rate in existing research and industrial setups. This software is able to
48

Chapter 4. Practical Implementation 4.3. Implementations of Froth Surface Analysis
sample at periods that are an integral multiple of 20ms, whereas the customary sampling rate for
industrial setups is 30 to 90 seconds. The results of this novel experiment are detailed in chapter
5.
50

Chapter 5
Performance and Impact
This chapter details the performance of the flotation froth analysis algorithms and the impact of
their use on real flotation plants. The difficulty of conclusively validating algorithm performance
is explained and investigations into alternative measures of accuracy are documented. Experi-
ments with continuous video flotation analysis and their pertinence to algorithm validation are
documented. Subsequently, the performance of the algorithms in real-world flotation-plants is
discussed. Naturally, this practical measure is a very good indication that the underlying algo-
rithms are performing well in their function as measurement component of the control system.
5.1 On Algorithm Validation
Quantifying the image processing accuracy of the flotation algorithms is a very difficult task, as
there are no benchmark algorithms to compare with nor existing databases of froth images and
“correct” image parameters. Creating such a database is not in the scope of this work, although it
would be a very valuable asset in the light of current developments in flotation froth analysis. An
alternative is to compare the algorithm performance to that of a human vision system. This is also
not viable, as visually inspecting a statistically significant sampling of images and comparing
the result of this inspection with the algorithms’ performance on the sampling would take a
prohibitively long time.
5.1.1 Experiments with synthesised motion
Another method of testing computational vision algorithms is synthesising data with known pa-
rameters and then testing the algorithm extracted parameters against the known ones. Once again
51

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
0 motion of a greater magnitude, more incorrect bubble-motion vectors would be generated in
regions with smaller bubbles.
The maximum detectable magnitude of motion detectable by the region-based motion estimation
is limited by the size of the search area. This size is adjustable however. Although making it
larger increases accuracy, the size increase increases execution time significantly.
5.1.2 Qualitative judgement
Extensive qualitative measurements of the algorithms’ performance were made by examining
its graphical outputs both on the extraction plant and in the laboratory with hours of recorded
flotation froth video footage. The graphical outputs are segmented images such as that in figure
2.8 and images with calculated motion vectors overlaid such as that in figures 2.13 and 2.14.
Judging by these observations, the algorithms perform satisfactorily.
5.2 Continuous Sampling Experiment
When the froth surface analysis software is in use on the plant, typical image sampling periods are
at the very least 15 seconds, more often 30 to 120 seconds. This is also the norm for JKFrothCam
[16]. This period is very short in comparison to the time constants of processes on the plant where
a disturbance is considered very fast if it lasts an half-hour. Because of this, no work has been
published on the analysis of continuous sampling and analysis of froth surface video data. In the
course of this work, some investigations were made into this.
Video from a laboratory flotation cell and from four industrial cells in a South African platinum
extraction plant was used. Segments of 16 seconds (representing 400 images at 25 images per
second) were digitised, as well as longer segments of 2 minutes (representing 1500 images at
12.5 images per second). Full static and dynamic analyses were performed on some 19200
images. The continuous sampling platform of section 4.3.4 was used to perform the digitising
and analysis. Interlaced images of dimensions 380x284 pixels were captured, of which only the
even fields were used for these analyses. The region of interest for the area-based motion analysis
was chosen on the grounds of the location of the weir which varies for our datasets, but was kept
to a third of the image area. The effective partitioning (motion block) size for the region-based
motion estimation was chosen as 10x10 pixels and the search area as a 16x16 congruent block.
On an entry-level Intel Celeron 366MHz computer, a full static analysis on one image and a full
dynamic analysis on that image and the next takes on average 2.5 seconds.
On the grounds of the investigation of the analyses results, several important observations can be
made.
53

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
144 145 146 147 148 149 150 151 152 153 154
450
500
550
600
650
700
750
Bubble Count
Bub
bles
Image
Figure 5.1: Small section of number of bubbles plot for 25 image per second sample set of laboratory flotation cell
froth illustrating number of bubbles sensitivity.
5.2.1 Continuous Video Static Analysis
In this section, we will look at the parameters extracted from images taken of flotation froth in a
laboratory environment. This froth is quite unstable and facilitates the investigation of selected
froth dynamics and events which aren’t so readily available in controlled froths in industrial
environments.
The bubble count and average bubble size (by its inverse relation to number of bubbles) can
show relatively large jumps for very little humanly visible change between two images. If it
is not immediately apparent why a parameter experiences a discontinuity, examination of other
parameters often yields the answer, and this will shortly be demonstrated.
For example, figure 5.1 shows a small section of the number of bubbles parameter plot for a 25
image per second sample set of the laboratory flotation cell froth and figure 5.2 shows some of
the segmented images corresponding to the graph.
Image 144 shows an image containing a few very large bubbles and a majority of very small
bubbles. Figure 5.3 then shows a sharp acceleration in vertical motion which is reflected in the
ellipticity plot (the average ellipticity increases because of the froth’s natural cohesive resistance
to the acceleration) in figure 5.4 and in the froth image 145. Probably because of this sudden
perturbation, the large bubble on the right starts to weaken and is drained of its mineral load.
Note how the segmentation reacts to the start of the bubble-burst. In image 146 this large bubble
bursts and the bubble count increases dramatically with a corresponding decrease in ellipticity
(bubble tension is decreased by the large bubble burst) and in vertical speed. A brief steady state
54

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
144 146 148 150 152 154 156
0
2
4
6
8
10
Average Vertical Motion
Image
Mot
ion
[Pixe
ls]
Figure 5.3: Section of average vertical pixel motion as calculated by region-based motion calculation for 25 image
per second sample set of laboratory froth. Compare with the images in figure 5.2.
is reached at image 148 as the froth almost comes to a standstill and ellipticity falls back to its
initial value. By image 149, the froth accelerates once again and this is reflected in the increased
ellipticity, decreased bubble-count and increased vertical motion.
From this example and from other investigations on the continuously sampled images, we see
that most parameters that are derived directly from the bubble segmentation suffer from a very
high sensitivity to froth image perturbations because of the high accuracy of the segmentation
that we use. In other words, froth surface images that represent froths that might seem very
similar to a trained froth operator yield significantly different extracted parameters.
This is a positive observation, as it is a more correct approach to extract information from ac-
curate parameters that convey physical significance than it is to extract information from under-
sensitive parameters which might at first glance seem more directly related to the high-level froth
condition and more stable. More simply put, the latter approach could already discard important
information at the data-gathering stage, rather than doing so selectively at the decision-making
stage. This is very important, especially in the light of the fact that other froth analysis ap-
proaches such as the one employed by JKFrothCam [16] deduce froth type from only textural
information. It is also in line with the proposed classification approach in [7] where it is stated
that images should not be analysed as patterns only, but rather that extracted measures should
convey real physical significance.
By employing a mapping function such as a filter or a higher-level classifier, the more accurate
parameters can be translated during the decision-making stage to more stable signals representing
higher-level characteristics, such as the froth type or condition. It is of primary importance that
56

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
144 146 148 150 152 154
5
6
7
8
9
10
Ellipticity Average
Image
Ellip
ticit
y
Figure 5.4: Section of ellipticity figure plot for 25 image per second sample set of laboratory froth. See figure 5.2.
the features used for these transforms are as accurate as possible.
This example also serves to demonstrate the detail in which froth bubble dynamics can be exam-
ined by using continuous sampling methods. By sampling images at continuous video rates and
extracting visual parameters with these algorithms, the physical behaviour of flotation froth can
be examined at more depth than possible before.
5.2.2 A Useful Filter for Froth Parameters
Figure 5.5 shows a plot of extracted average bubble area over 400 images of on-plant tank cell
froth. The images were captured at 25 images per second and represent 16 seconds of full motion
video. Note that even over this relatively short period we see the average bubble area parameter
ranging from 490 pixels to a peak of 1950 pixels. The 10-90% interval is at 660 to 1200 pixels,
which is still very significant for this short period. The images representing these peaks will
shortly be examined.
Note that the extracted parameter plot seems very noisy, although there is a clearly discernible
trend. This noise is a complaint often cited by process engineers when faced with the problem
of using these signals for control purposes. From investigations of sequences of segmented data,
we know that this “noise” is an accurate reflection of occurrences in the captured images. In this
case, the “noise” is exacerbated by the fact that the framing of the images includes the flotation
cell weir in order to perform motion analysis on froth moving over the cell lip.1 By the nature
1Build-up on cell lips can affect recovery. Monitoring mobility over the cell lip is an effective way of monitoring
57

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
0 50 100 150 200 250 300 350 400400
600
800
1000
1200
1400
1600
1800
2000
Image
Ave
rage
Are
a [Pi
xels]
Figure 5.5: Plot of average bubble area/size over 400 images representing 16 seconds of on-plant tank cell froth
digitised at 25 images per second.
of its function, bubbles drop away from the camera over the weir, thus causing significantly
different segmentations at every image capture and significantly different resultant parameters.
The lighting conditions at this point can also be worse as will shortly be seen in the sample
images.
A useful filter for extracting the parameter trend is the α-trimmed mean filter [28], which is a
member of the class of filters known as L-filters. L-filters are order-statistic and by implica-
tion non-linear filters that make a compromise between pure non-linear filtering and pure linear
filtering. The output of the L-filter is given by:
L(X1,X2, . . . ,XN ;a) =
N
∑
i=1
aiXi
where X1,X2, . . . ,XN are the signal values in a moving window sorted in ascending order of value
of size N and
a = (a1,a2, . . . ,aN)
is a weight vector.
The α-trimmed mean filter is defined as:
TrMean(X1,X2, . . . ,XN ;α) =
1
N −2αN
N−αN
∑
i=αN+1
Xi
where α is the fraction of samples that get trimmed off the start and the end of the sorted moving
window. In other words, αN samples are discarded from the beginning and the end of the sorted
build-up.
58

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
0 50 100 150 200 250 300 350 400400
600
800
1000
1200
1400
1600
1800
2000
Image
Ave
rage
Are
a [Pi
xels]
Figure 5.6: Average bubble size signal filtered with α-trimmed mean filter with N = 10 and α = 0.2
moving window, and the output of the L-filter is the numerical average of the remaining values.
Larger α values result in a closer resemblance to a median filter and smaller values result in a
closer resemblance to a mean filter.
A median filter discards impulse noise well, but is not well-adapted for additive Gaussian noise
removal, whilst the mean filter handles additive Gaussian noise well, but fairs badly with im-
pulsive noise [28]. The trimmed mean filter is a compromise which yields good results in cases
where impulsive noise and Gaussian noise are present. In our case, there is impulsive “noise”
which can be attributed to the sensitivity of the segmentation-based parameters and we assume
that the segmentation error can be modelled as Gaussian noise.
Figure 5.6 shows the signal in figure 5.5 after having been filtered with an α-trimmed filter with
N = 10 and α = 0.2.
As a quick illustration of the segmentation performance, the segmentations of images 117, 132
and 152 are shown in figure 5.7. The first image represents the highest peak of the bubble size
plot in figure 5.6, the third image represents the deep trough immediately following it, and the
second image is halfway between them. From the input images the large drop in average bubble
size is clearly visible.
5.2.3 Continuous Video Dynamic Analysis
The average horizontal or vertical motion is the numerical average of the applicable rectangular
components of the calculated vector field. The vector field is the array of motion vectors gen-
59

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
Figure 5.7: Input and segmented froth images corresponding to graph in figure 5.6. Images 117, 132 and 152 from
top to bottom.
60

Chapter 5. Performance and Impact 5.2. Continuous Sampling Experiment
0 50 100 150 200 250 300 350 400−6
−5.5
−5
−4.5
−4
−3.5
−3
−2.5
−2
−1.5
−1
Image
Hor
izon
tal M
otio
n [Pi
xels]
Figure 5.8: Plot of average horizontal pixel motion as calculated by the region-based motion estimation over 400
images sampled at 25 images per second from on-plant scavenger cell. The darker line is a version
smoothed with an α-trimmed means filter with α = 0.2 and N = 10.
erated by the region-based motion estimation, where one motion vector is calculated for each
partitioned motion block.
Figure 5.8 is a plot of the region-based motion estimation calculated average horizontal motion
over 400 images sampled at 25 images per second from an on-plant scavenger cell and 5.9 is a
plot of the per image standard deviation of this parameter. This sampling of the scavenger shows
a relatively continuous horizontal flow over 16 seconds and the standard deviation values are
relatively small in comparison to the average horizontal motion values that they correspond with.
Examining the sample images also show that the motion vectors calculated for each image are
generally in full agreement, i.e. they are all of almost equal magnitude and pointing in the same
direction. Keep in mind that each vector represents the motion of a separate independent parti-
tioned block of the motion region of interest. This behaviour is to be expected of a flotation cell
which is kept under effective control. There are no major step discontinuities in the motion and
the standard deviations are relatively small indicating that the region of interest is experiencing
constant translational motion.
In order to demonstrate the behaviour of parameters extracted from froth which is not so well-
behaved, a sample set of images captured of laboratory laboratory froth was analysed. Figures
5.10 shows a section of the average vertical motion plot and figure 5.11 shows the corresponding
per-image standard deviation.
This section has been chosen to illustrate how average motion standard deviation and average
motion itself reflect froth events such as bubble bursts, and indirectly froth stability. At image
357 the standard deviation is relatively low in comparison the the vertical motion. The first
61

Chapter 5. Performance and Impact 5.3. Performance and Impact on Flotation Plants
frames per second), obviously depending on the particular dynamics of the froth under investi-
gation. This would enable the estimation of very accurate yet stable parameters. If more than
just stable parameters are desired, i.e. alternative methods for e.g. froth stability quantification,
the recommendations in the following paragraphs should be followed.
Continuous video froth analysis can yield more information than the usual image sampling about
the flotation process which is being monitored, as temporal relationships between information
gathered from successive frames can be utilised for more than just stabilising existing parameters.
As an example of this, changes in bubble count between successive images can be used as a more
accurate and intuitively pleasing measure of froth stability. As we’ve seen in this investigation,
bubble burst events can cause very large perturbations in the continuous video bubble count
parameter. As demonstrated, a mobility analysis where motion and motion standard deviation
are taken into account can also be used for stability calculation.
This investigation has shown that although process disturbances have relatively long time con-
stants, there are important froth surface appearance dynamics with lifetimes of as little as tens of
milli-seconds. The sampling scheme should be adjusted if these dynamics are to be accounted
for.
The summarising recommendation is that the sampling scheme for practical analysis setups
should be changed if the full information content of the extracted parameters is to be utilised.
Again depending on the particular froth dynamics under investigation, it is recommended that
the sampling period is increased drastically, but that 2 or 3 seconds of continuous video or half-
rate continuous video (12.5 images per second) is sampled at each sampling interval. In this
way, the sampling scheme benefits from the long time constants of the process disturbances
themselves, but also detects the low-level froth dynamics, enabling the extraction of accurate
and stable parameters as well as the extraction of hitherto unavailable froth information, such as
true stability.
5.3 Performance and Impact on Flotation Plants
Recall that froth surface analysis is traditionally performed by human operators who don’t re-
port on the number of bubbles they see or the average size, but rather on the deductions they
make from their visual observation. These froth operators are judged on the flotation process
performance which is a result of the decisions and deductions that they make.
The Fluxar Control System has been running in an advisory fashion on a South African platinum
extraction plant since the end of 1998. Only relatively recently have tests started with full closed-
loop control by Fluxar, so conclusive results with respects to increased recovery or grade are not
yet available. The system is being tested in close-loop control mode every alternative day. On
the non-control days it functions in an advisory mode where instead of performing adjustments
65

Chapter 6. Conclusions 6.2. Contribution
applications. It is able to perform a full static and two-image dynamic analysis in a few sec-
onds, depending on the capturing and analysis parameters. Literature does not mention a single
complete implementation such as this.
Three software packages incorporating this froth analysis kernel were developed. The most
extensive one, Fluxar-SP, affords its user easy and user-friendly access to all the froth analysis
algorithms and post-processing functionality through a graphical user interface. It is able to
acquire image data from four multiplexed video input channels so that it can be deployed in
an on-line fashion on flotation plants as the measurement component of an automatic control
system. It is also able to acquire data from images stored on disc so that froth images can be
analysed off-line. This enables it to be used for after-the-fact analysis of process events as well
as a low-cost research tool. Fluxar-SP has proved to be a robust application and has shown in
practice that it can fulfil its purpose as the sensor component of an automatic flotation control
system. It is in use in selected research setups and in an industrial flotation plant.
Fluxar-Speed is an embedded application of the froth analysis system and is used on plants
where a low-cost mobility-monitoring device is required. Because it only has to extract and
output mobility information, it can do this at the rate of 10 or more samples per second. This
tool is in the process of going to international market.
The Continuous Sampling Platform is a research tool that was designed to capture large batches
of continuous video data and analyse them off-line. It performs all available static and dy-
namic analyses on the captured data and stores all extracted information for experimental post-
processing and investigation.
The performance and impact of the algorithms were then investigated. The difficulty of theo-
retically quantifying froth analysis algorithm accuracy was explained and an attempt was made
at partly validating the mobility analyses. A significant preliminary study was done on selected
issues pertaining to continuous video flotation froth analysis and it was shown that this is a very
promising avenue of research for froth visual analysis. Using the video data that was captured,
an important issue of parameter stability vs. parameter accuracy was also discussed. The chapter
concluded with a short discussion on the performance and impact of the froth analysis system on
flotation plants.
6.2 Contribution
This work yields several important contributions to the froth analysis industry:
1. This work provides a knowledge base for further research and development in computa-
tional vision flotation froth analysis specifically, extending the more general knowledge
base for machine vision control of flotation provided by Eksteen’s work [1].
68

Chapter 6. Conclusions 6.3. Future Work
2. It includes the most complete froth surface visual analysis implementation known to exist.
This is in the form of a reusable and portable kernel that can be re-deployed in any number
of froth surface analysis applications. In addition, a full static (including complete bubble-
segmentation) and dynamic analysis can be performed in a few seconds.
3. Three practical applications have been created. Fluxar-SP is a complete and operator-
friendly front-end to the froth analysis kernel for on-plant and research use. Fluxar-Speed
is an embedded mobility monitoring device that will find widespread industrial use. The
continuous sampling platform is a valuable research tool for investigations into an inter-
esting new direction.
4. Fluxar-SP is installed in a factory and although no conclusive results concerning its per-
formance are available, it is performing very satisfactorily as a robust and accurate froth
analysis platform. It is important to note that this development took place in close coop-
eration with the froth operators and chemical engineers who make use of it, and is thus
well-adapted for practical use in industrial environments.
5. Novel additions to froth analysis algorithms:
• Two methods for fine froth detection in froth images.
• A heuristic method for post-segmentation over-segmentation correction.
• An algorithm for the extraction of per-bubble motion.
• The first application of forget-factor based filtering to froth parameter histograms
6. A preliminary investigation into continuous video froth analysis was performed and se-
lected issues were discussed. There is no literature on previous experiments of this kind.
The implementation infrastructure for our investigation can be re-used for further research.
6.3 Future Work
This work opens up several new and interesting opportunities for other research. From an image-
processing perspective, a very valuable contribution to the froth surface analysis field would be
the creation of a reference database of images that also contains human froth-operator and chem-
ical engineering information describing each image’s significance to the process. This database
could be used to benchmark froth analysis and classification algorithms, as well as being a very
valuable tool in operator training.
The analysis software developed as part of this work facilitates the investigation of various clas-
sifiers and other pattern-recognition constructs for the automatic extraction of process variables
from imaging parameters. Up to now, most work has been done on classifying froth types on the
grounds of texture-related information. Now we have access to parameters representing physical
69

Bibliography
[1] J. J. Eksteen, “The Development and Technology Transfer of an Industrial Machine Vi-
sion System for the control of a Platinum Flotation Plant,” Master’s thesis, University of
Stellenbosch, February 1995.
[2] V. A. Glembotskii, V. I. Klassen, and I. N. Plaksin, Flotation. Primary Sources, New York,
1972.
[3] A. J. Lynch, N. W. Johnson, E. Manlapig, and C. G. Thorne, Mineral and Coal Flotation
Circuits, vol. 3 of Developments in Mineral Processing. Elsevier Scientific Publishing
Company, 1981.
[4] D. W. Moolman, C. Aldrich, G. P. J. Schmitz, and J. S. J. van Deventer, “The interrelation-
ship between surface froth characteristics and industrial flotation performance,” Minerals
Engineering, vol. 9, no. 8, pp. 837–854, 1996.
[5] J. Villeneuve, J. C. Guillaneau, and M. V. Durnace, “Flotation modelling: A wide range of
solutions for solving industrial problems,” Minerals Engineering, vol. 8, no. 4/5, pp. 409–
420, 1995.
[6] M. Bezuidenhout, J. S. J. van Deventer, and D. W. Moolman, “The identification of pertur-
bations in a base metal flotation plant using computer vision of the froth surface,” Minerals
Engineering, vol. 10, no. 10, pp. 1057–1073, 1997.
[7] D. W. Moolman, J. J. Eksteen, C. Aldrich, and J. S. J. van Deventer, “The significance
of flotation froth appearance for machine vision control,” International Journal of Mineral
Processing, vol. 48, pp. 135–158, December 1996.
[8] D. W. Moolman, C. Aldrich, J. S. J. van Deventer, and W. W. Stange, “The classification of
froth structures in a copper flotation plant by means of a neural net,” International Journal
of Mineral Processing, vol. 43, pp. 193–208, 1995.
[9] D. W. Moolman, C. Aldrich, J. S. J. van Deventer, and D. J. Bradshaw, “The interpretation
of flotation froth surfaces by using digital image analysis and neural networks,” Chemical
Engineering Science, vol. 50, no. 22, pp. 3501–3513, 1995.
71

Appendix A. Circular Statistics A.1. Selected statistic measures
P
O
Q
Figure A.1: Illustration of median direction
A.1.2 Median Direction
The median direction ζ0 of a set of sample angles is any solution of
∫ ζ0+pi
ζ0
f (θ)dθ =
∫ ζ0+2pi
ζ0+pi
f (θ)dθ = 1
2
where f (θ) satisfies f (ζ0) > f (ζ0 +pi).
In more graphic terms, if all the angles in the sample set are considered as points Pi on a unit
circle, any one of these, P, is considered a median if the line PQ (with Q being diametrically
opposite P) has an equal number of sample points on either side, and the majority of the sample
points are closer to P. See figure A.1.
For data on a line, it is sufficient to perform sorting and pick the middle element. In the circular
case, it is easy to see that this can yield incorrect results. For example, consider an example from
our references with sample angles 43, 45, 52, 61, 75, 88, 88, 279 and 357 (in degrees) where
the median direction according to the above definition is 52, whilst for the case on the line it
would’ve been 75.
Note that for data falling in the interval 0 to 180o, the above definition simplifies to sorting and
selecting the middle element.
A.1.3 Circular Variance and Standard Deviation
Again representing sample angles with points on the unit circle, consider P representing the zero
direction, and Pi a sample point, then a measure of dispersion ζi of the sample point from the
point representing the zero direction could be defined as
ζi = min(θi,2pi−θi) = pi−|pi−θi|
i.e. the smallest angle between the two vectors OP and OPi.
75

Appendix A. Circular Statistics A.2. Angular Distributions on the range (0,2pi/l)
A.2 Angular Distributions on the range (0,2pi/l)
Consider a random variable θ that is entirely on the range (0,2pi/l), i.e. regard it as only having
values in that range. Define
θ′ = lθ
which will have values on the interval (0,2pi).
Now, if we define x′0 and s′0 respectively as the mean direction and circular standard deviation of
θ′, then the mean of θ
x0 = x
′
0/l
and the deviation of θ
s0 = s
′
0/l
Defining the translated variance is not as easy. For a distribution concentrated in a small arc
about the mean, the relation
V0 = V ′0/l2
works well. In other cases, for wrapped normal and Von Mises distributions (see the reference)
Vo = 1− (1−V ′0)1/l
2
is a valid general relation.
For the median direction, the method documented in section A.1.2 can be used as is, no angle
translation has to be done.
77