Workshop: Digital Image Processing like a boss [Lesson 02]

[Novacaptain]

Lesson 1 - setting up.
is posted. Please submit your results and any questions/comments if you have any. I'll start working on the next lesson as soon as there is any activity. I could probably squeeze in more than one of these per week if you have the time and interest.

Lesson 2 - Dissecting a grayscale image.
Where we load an image from a file, spill its guts out and then learn some simple photoshop tricks like inverting and how blending modes work.


Hey everyone,

I'm back in school, getting my master's degree in robotics and I've just started a course in digital image processing. I find it super-interesting because I'm getting to learn the mechanisms behind what I had previously known as gaussian blur, autolevel, median filter, find edges, etc. I was thinking that there might be some interest here in starting up a small on-line workshop to go through some of these filters and maybe talk a little about digital images in general.

Taking it one step further:
We can use a freely available (and easy to use) programming language such as octave to actually design some of those well-known filters together, and maybe invent some completely new ones.

Going all the way:
Take the red pill or the blue pill? The choice is yours.

Possible expected results:
Getting to know the bits and bytes behind digital images, and better understanding the effects of addition, subtraction and multiplication of images.
Understanding filters from the core, histograms (levels), curves.
writing your own programs to process batches of images or videos like a wizard.

Anyway, since I suspect a few of you already do stuff like this, perhaps you'd like to contribute on the "teaching" side? What are the things you'd be most interested in learning? Is anyone curious at all about looking under the hood or are you happy knowing your tools on an intuitive level? Driving a car doesn't require that you be a mechanic, after all ;)

cheers.

[Pieter]

I'm pretty interested in stuff like this. Although I can't even make a hello world program compile.
I love the techniques and theories behind it though, something about an effective algorithm that tickles my nerd lobe.

I also know there's a 'custom' filter in photoshop where you can do the kernels by hand, I'dd be happy if I could understand that one at minimum.

[Novacaptain]

Oh, why didn't I think of that? Good idea! :)

[road]

i would be interested in this for sure!!

[Pootivic]

Man it sounds fascinating, but I don't know a lick of programming! Fuck though, I'd really like to know how those worked. It would all depend on accessibility and cost for me.

[Novacaptain]

eh, think of it more as a forum activity kind of thing. And programming in octave/matlab is stupid-simple due to all the built-in high-level functions. It's like writing a recipe for cake..

BTW octave is free

[Pootivic]

You're saying all the words I want to hear

I'm totally down, I don't know how to really give feedback on what to teach. All my "computers" courses in art school were mostly telling you what the programs did, not how they worked. Its one of the reasons I probably looked like a tool in that thread I started, trying to discuss how pre-digital art theory translates into a digital medium.

[Djävulen]

I think pixelbender is pretty easy to understand, if you want to make new blendmodes and stuff. And it obviously works well with photoshop.

[Pieter]

Hey Nova, still interested in this?
I'm having my exam period so I probably won't be able to do anything the next month or so.
But I'm still definitely interested.

[Novacaptain]

yeah, im definitely up for it! I'm taking my final exam in this subject on monday and a couple of weeks after that I start working at my old job for the summer, this is a typical 9 to 17 job though rather than the 24-hour madness (oh wonderful madness) of school. I've been thinking a lot about this workshop idea, mostly on the scope and complexity of it. I think it should be kept on the practical side of things.

Djävulen: Ever since Photoshop CS 3 (extended), there is an integration with matlab (which is an amazingly powerful tool) and photoshop (which is an amazingly powerful tool). I feel that this and the fact that we can do a non-proprietary, open-source workshop through octave makes it a good choice. Octave and matlab are for the most parts identical in syntax and functionality. So learning one is the same as learning the other.

see http://www.inthemod.com/bps/?p=136 for more info on the integration between PS and matlab

A basic outline of what I have in mind is:
- How images are represented in a computer
- Basic Octave / matlab intro (easy, i promise)
- Gray-level transforms (basic math on a per-pixel basis, or more advanced if desired)
- More on octave/matlab (or i provide you with functions that you can use as a black-box, but look into if you want to)
- Spatial filtering (considering a neighborhood of pixels. This covers many of the blurring, sharpening, edge finding filters)
- Frequency-domain filtering (ok, this is REALLY cool stuff. If a digital image was a matrix [and it is!] looking at it in the frequency domain is like being Keanu Reeves in a trench-coat. The math might be a little tricky here, but it's something we can skip while still reaping large benefits from learning on just an intuitive level)
- Color image processing

I think this perhaps enough to start with and should give you something to sink your teeth into. Small programs that create images can be considered too, or ways of extracting features from an image, such as the texture.

[Pootivic]

I feel super excited for this. I feel guilty that I have no money to pay you and that you're not asking for any.

[SLiD3]

bring it dani BRING IT!

[Novacaptain]

hahaha wp, the likeness is uncanny

Ok let's get started.

Step 1.
Download and install octave:

Windows users:
Find your installer at*
http://octave.sourceforge.net/
Run the exe, if the installer offers to include gnuplot agree to this.

Mac users:
Find yourself a disk image (dmg) at*:
http://octave.sourceforge.net/
mount the image by double-clicking it and drag the octave icon into your applications folder
then go to the Extras folder in the virtual disk and find the gnuplot dmg. Mount this one and install gnuplot by dragging the gnuplot app into the applications folder.

Linux (ubuntu) users can find octave in their software center or install through the command-line using:

Code: Select all

sudo apt-get install octave

Completing this step earns you the achievement: Free Software FTW!

*Note about versions: I'm using version 3.2.4 but there are newer versions (3.4 at time of writing). If you want to use a newer version, go ahead but it's a bigger download (and I won't make use of any feature past 3.2.4 anyway). It should be backwards compatible though, so if you do pick a newer version it should work just as well.

Step 2.
Run octave, either by clicking on an icon or by typing the appropriate commands in a terminal-window of your operating system. Regardless of how you do it, the result is that you'll be looking at something like this:

Windows


Mac


Linux (for example)


To test that the installation has been done successfully up to this point we will do two simple tests.

type, followed by <enter>:

Code: Select all

x=0:0.1:8;

This means: I want all the numbers between zero and eight, in steps of 0.1 to be stored in a variable called 'x'.

The semi-colon at the end of the line will keep octave from printing all the numbers on the screen. If you want to see which numbers they are (0 0.1 0.2...), you can of course leave out the ";".

If you decided to leave out the semi-colon, and are done inspecting your numbers, press "q" to return to the command line.

now type, followed by <enter>

Code: Select all

y=sin(x);

This means: For every value of x, I want you to compute the sine and store the result in a variable called 'y'.
the Sine function is built into octave, so octave knows what to do when it gets sin(x) (as long as x is defined).
You can type

Code: Select all

help sin

this spits out some basic instructions on what the function does, in case you forget. If you need more help understanding a certain math function, wikipedia or wolfram|alpha are great places to look.

but how do we know what variables are defined? There is a command for that too:

Code: Select all

whos

This tells us who the defined variables are. Another way to use it is:

Code: Select all

whos y

if we want to know specifically who y (for example) is.

The grand finale of step 2 (and this very basic install octave lecture) is to do the following:

Code: Select all

plot(x,y)

screencap and post your results (or your errors) for the "mommy i r hax0r" achievement.

[road]

downloading now..

[Pootivic]

Edit:

[Novacaptain]

Ok, so it looks like some of you have either started or finished the prevous step.
Let me know if there are any things you'd like to explore further or if you'd like to skip something in favor of something else. feel free to ask questions or just join the discussion in any way you want.

Anyhow, what I have in mind for today's lesson is to talk about what's behind the pixels we see on our screen.

So, as a first step, start octave.

Since we will be working with images, we need to be able to find files in our computers...since images are files, after all. If you're familiar with DOS or the linux shell, just scroll past this section.

when in octave, type

Code: Select all

pwd

This command will print your working directory. Files in your working directory can be accessed directly, without needing a full path specified in front of the file name. This means that instead of having to type '/users/daniel/pictures/apedogs/ball.png' I could type only 'ball.png'. You can change your working directory using "cd" as you would in DOS.

To show the contents of the current directory use "dir" or "ls" (equivalent). "cd .." takes you to the parent directory. "cd dir_name" takes you into the directory called "dir_name" if it shows up in the list.

one last thing about directories, no matter which directory you're in, octave can always find the files containing its functions (like sin, that we used last time). This is because octave has a list of places where it always looks. This list can be seen by using typing

Code: Select all

path

. If you will use a directory often, you can add the path to this directory by using the

Code: Select all

addpath

command. Help on all the commands I've mentioned here can be accessed by typing "help" followed by the name of the command.

phew! That was even boring for me, but I hope you're still with me, because now comes a little bit of fun stuff.

I would like you to save this image-file in a folder on your computer.

now, using octave, change the current directory to the one where the image is saved. Then type:

Code: Select all

I=imread('ballgray.png','png');

This uses a built-in command called "imread" which reads an image from a file and outputs data (which we asked to have stored in a variable called "I"). The name of the file is ballgray.png and the format is Portable Network Graphics (png). Specifying the format is not necessary in most cases, but we do it here to help you figure out what to do in case octave doesn't automatically recognize the format some day.

So we read an image, what now? let's take a look at the variable that was created.

Code: Select all

 whos I

this tells us that:
-The name of the variable is I,
-it has a size of 15x14
-it contains 210 bytes
-and it is of class "uint8".

I figure you can guess what most of this means without me telling you, with the exception of the class, so let's talk about that. uint8 means "unsigned 8-bit integer". unsigned means that there is no "sign" ie. no plus or minus. So data in this class can only be positive.

integer means that we only have whole numbers. 1, 3 ,4, 5000 are integers. 3.1415, 1/2 etc are not. Trying to store a number like 9.9 in a variable of the integer class will result in 9 being saved and 0.9 being ignored.

ok, so we have positive integers, what does the 8-bit tell us?

The 8-bit tells us that we have 8 bits (one byte) to represent the numbers. in octave, do

Code: Select all

14*15

.
210, right? and how many bytes did "whos I" tell us? 210. That means that each pixel in the image contains one byte of data.

but what does it MEAN!? hehe
well, there are 10 types of people in the world. Those who understand the binary system, and those who don't.
but fear not. you will get it soon enough.

in the decimal system 10 means ten.
in binary 10 means two.

in decimal
1111 means one thousand, one hundred, ten and one, in other words: 10⁰ + 10¹ + 10² + 10³
in binary
1111 means simply : 2⁰ + 2¹ + 2² + 2³

so when we're told that there are 8 bits storing each value we can have any value between
00000000
and
11111111

type (or copy) into octave:

Code: Select all

2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + 2^6 + 2^7

that's right, 255. suprised? didn't think so

in fact, if you simply type

Code: Select all

I

you'll get the numeric contents of the image printed on your screen.
Can you imagine what the image looks like? Where are the brightest and darkest parts?
What value would a 50% gray have?

let's look at how the computer translates this image to pixel brightness:

Code: Select all

imshow(I);

imshow is another built in function that displays a matrix as an image.

ok so far? then let's do one last thing.
to invert an image:

Code: Select all

I_inv=255-I;
imshow(I_inv);


OR, directly:

Code: Select all

imshow(255-I);


why? because when a pixel in I is white (255) the difference between 255 and 255 will be 0 (black).
when a pixel in I is black (0) the difference between 255 and 0 will be 255 (white). Everything in between will follow the same logic. However, since there are an even number of values from 0 and 255 there is no true 50% gray. If you had a pixel with value of 128, the inverted image would contain 127 and vice-versa.

Optional bonus-quests:
Load a color-image with imread and do a "whos". Notice any differences? of course you do.

if you create two equal-sized images, you can now experiment with the photoshop layer blending modes by adding, subtracting and multiplying them.

Code: Select all

I1=imread('firstimage.png');
I2=imread('secondimage.png');

figure(1)
I_added=I1+I2;
imshow(I_added);

figure(2)
I_subtracted=I1-I2;
imshow(I_subtracted);

figure(3)
I_multiplied=I1.*I2; %the "." makes a pixel-by-pixel multiplication.
imshow(I_multiplied);


[Pieter]

playing with ma kittenz and them pixelz

[Novacaptain]

awesome!

by the way, to save an image use the following command:

Code: Select all

imwrite(I,'filename.jpg','jpg')

Like before, the last 'jpg' is not strictly necessary, but can be useful if you want your files ending in jpe or jpeg instead of jpg and octave doesn't recognize the extension. other file formats such as bmp, tif, gif, png are also possible.

if you want files saved somewhere other than your working directory you can either change the current directory or specify a path in front of the filename.

also, code such as:

Code: Select all

imwrite(255 - imread('input_image.png'),'inverted_image.png')

is also possible. it reads, inverts and saves the image directly without bothering to store the data in variables explicitly.