baiban

how to use:

• start in lower left corner
• click outer corners of whiteboard going clockwise
• refresh to clear

Baiban is 'whiteboard' in manderin. This was my project to try and make a web app that would scan whiteboards directly, but it would work for other documents too.

I wanted to make it easier to take a photo of a whiteboard or blackboard, because I sometimes take notes that way. Apps like this exist, but not any that exist in the browser alone. I wanted to make it so I could use GLSL to do the image procesing quickly and on the user's device.

The components involved would be:

• get live camera grab from phone camera
• have user draw on canvas, and convert that to a poly
• get corner locations from the poly
• sort corner locations
• use corner locations to infer target polygon, as well as color to mask out (marker color)
• send that information to GLSL shader to apply transform to each pixel and apply mask with eculidian color difference
• supply image for download

I made javascript and shader code which allows a user to click the corners of a predefined image in order to choose what to 'scan.' Then, that information is sent to the shader to apply a transformation. Sometimes a transformation looks pretty good, other times the entire canvas is turned into a messy blur and I can't tell where the selection scanned is, although it is probably somewhere off canvas

I'd like to implement the rest of the stuff, I want to make it so the selected transform is centered in the canvas first. One approach may involve doing some hand waving to properly zoom and offset the canvas so that the translated selection is centered. I already tried altering the target poly, and found that it's most helpful when it's the same size of the selection. Changing the origin had little to no effect. Another method may simply be just 'pulling out' the pixels inside the selection, performing the trnaslation just on those, and sending the new pixels to a seperate WebGL instance or canvas. I'm not sure what this would look like given how hard it is to debug GLSL shaders, but this may be how other methods may do it. Probably the transformed space looks a little like a rect bounded by the 'target' poly?

• how to work with html5 canvas and webgl at the same time
• how to send lots of data to shaders
• what homography matricies are and how they work
• how to instantly load shader code into javascript from a seperate file

This article was pretty useful

home