how to use:
- start in lower left corner
- click outer corners of whiteboard going clockwise
- refresh to clear
Baiban is 'whiteboard' in manderin. This was my project to try and make a web app that would scan whiteboards directly, but it would work for other documents too.
I wanted to make it easier to take a photo of a whiteboard or blackboard, because I sometimes take notes that way. Apps like this exist, but not any that exist in the browser alone. I wanted to make it so I could use GLSL to do the image procesing quickly and on the user's device.
The components involved would be:
- get live camera grab from phone camera
- have user draw on canvas, and convert that to a poly
- get corner locations from the poly
- sort corner locations
- use corner locations to infer target polygon, as well as color to mask out (marker color)
- send that information to GLSL shader to apply transform to each pixel and apply mask with eculidian color difference
- supply image for download
I'd like to implement the rest of the stuff, I want to make it so the selected transform is centered in the canvas first. One approach may involve doing some hand waving to properly zoom and offset the canvas so that the translated selection is centered. I already tried altering the target poly, and found that it's most helpful when it's the same size of the selection. Changing the origin had little to no effect. Another method may simply be just 'pulling out' the pixels inside the selection, performing the trnaslation just on those, and sending the new pixels to a seperate WebGL instance or canvas. I'm not sure what this would look like given how hard it is to debug GLSL shaders, but this may be how other methods may do it. Probably the transformed space looks a little like a rect bounded by the 'target' poly?
What I Learned
- how to work with html5 canvas and webgl at the same time
- how to send lots of data to shaders
- what homography matricies are and how they work