here are running updates for sketch 3.
Wednesday March 27,
Today we presented our idea to the class about our Sketch 3, "Where the Light Follows". Here are some details about the project!
- Where the Light Follows
Team Roles:
- Anna & Kathy = dance / develop performance that complements the ideas of the show
- Doug and JJ = build system to track and project color images onto Anna and Kathy
- Trish = make videos/textures/colors to be projected
Project Concept/ Scenario:
- The idea of the project is to watch the girls dance and play and spread their cheerful light to the world around them
- The girls will dance as colorful images will be projected onto them.
In the room will also be a large, solid box on the floor that will have dull colors
Every time the girls touch the box (touch, sit, stand, lay ect) the box will change to a slightly
brightet color
Each time the box will stay a little brighter for a little longer
Concept Research:
- past experience helped us decide what to add and what not to add
Technical Research/ Tech Diagram:
- http://www.youtube.com/watch?v=tAGnSrdOfyA
- http://www.youtube.com/watch?v=-wVq41Bi2yE
- http://www.youtube.com/watch?v=5MXwJpKToEY
- Tech we will be using
* multiple kinect sensors (use pics from jj's slides)
* these will extract the depth data to give us the silhouette (use pics from jj's slides)
* Use the silhouette data to map texture/color to bodies and box
* will use openGL/C++ and Ogre for the actual programming/ rendering
- has a more correct depth buffer (zbuffer) than processing
*
Schedule:
- 1st 2 weeks: Dancers work on their dance and tech peeps work on the system
- 3rd Week : Start bringing the group together for final testing and final manipulation
Saturday March 30,
After several hours, JJ and I have successfully installed openCV and OGRE and are ready to begin working on the tech side of the project.
Monday April 1,
JJ and I completed the a small step in the project. We were able to correctly display a picture only on a silhouette and then used this progress to project the mapped visual into the OGRE rendering system. The progress can be seen in the videos below! The girls continued to work on their ideas for their dance and Trisha continued to work on the visual aspects of the project. This progress can also be seen below!
Wednesday April 3,
JJ and I began working on the actual projection of the images onto a human body. This turned out much harder than we expected. We are still working out the last few kinks but home to have it projecting correctly soon. The videos below show some progress. The girls continued to work on their ideas for their dance and Trisha continued to work on the visual aspects of the project. Looks like we are making some progress! Videos below.
Saturday April 6
JJ and I met in ETB today to work on the camera calibration between the Kinect and the projector. As it turns out it is a much harder task to complete than we thought! But we have set up a plan for how we can determine the transformation matrix between the two fields of views so we are not giving up yet!
Sunday April 7,
Today I finished writing a small program to capture 2D points by a mouse click and writing those coordinates to a file for use in determining the transformation matrix.
Given two matrices, one that contains n number of 2D coordinates and another that contains n number of 3D coordinates, we can find their transform as such
x1 x2 ... xn x1 x2 ... xn
Matrix P = y1 y2 ... yn and Matrix K = y1 y2 ... yn
1 1 ... 1 z1 z2 ... zn
1 1 ... 1
P = Transform *K
Monday April 8,
Today JJ and I added our parts together to try and pull together the Kinect-Projector calibration. We were very close to success but had a few little problems. We will be addressing these more on Wednesday to work out the little kinks. Here is a video explaining how the calibration works
Tuesday April 9,
Possible interactive art festivals to submit our work to!
- Santa Fe International New Media Festival
More to be looked into and added later!
Wednesday April 10,
Today JJ and I made some pretty good progress on the technical aspect of the projection mapping. However, due to our limited time on the project, we all made a team decision to change the project concept ever so slightly. We decided rather than focusing on extreme accuracy and making the projection mapping only on the dancers, we decided to make the projection some what of a shadow of the dancer. This way the dancer or audience user can play with their shadow while still having projection mapping on themselves.
Thursday, April 11,
Today JJ and I came to ETB and continued to work on the shadowing effect of the program. We also worked on making the video display only on the silhouette of the user. We were able to figure out the proper method for calibration between the Kinect and the projector and now to finish implementing it!
Monday, April 15
Today JJ and I finished most of the calibration formatting and were able to test it on the dancer who finished making their white costumes for the performance! Everything is coming together quite nicely. Only a few more tweaks to work on the calibration and we will be set. Also need to finish up the server code for receiving the information from the two Kinect system. A video of today's results is shown below.
Wednesday, April 17
Today, after a few hours of banging our heads against the wall, JJ and I finally figured out the trick to perfecting(or as close to perfecting that we will be able to get) the calibration system. We figured out the we needed 3 tables to use for the calibration system: two at diagonals and the other perpendicular to it. This allows us to get a more precise transformation matrix for the calibration! The results can be seen below.
Sunday, April 21
Today Peizhao gave us the confirmation that his 2 Kinect System has been successfully modified to send depth data over a UDP network as apposed to just the skeleton data. We will be testing it later today or tomorrow to mesh it with our code.
Monday, April 22:
This morning, JJ and I got together and set up the new computer that Peizhao built for the Kinect server/client system. We also set up the client end to work with our code. Here are a few pictures of the setup and us struggling with the calibration.
Well after a long night last night and a lot of work today we have gotten the 2 kinect system to work!!! Whoop!! Glad to be almost done!
Wednesday April 24,
Today JJ and I worked a little more on the final little kinks in the program. JJ was able to make the silhouette of the dancers be slightly larger than the dancer. This added a little more of a playful feel to the program. The dancer were able to demo the system as well. After the demo we decided we might need to add a third kinect. This shouldn't be to hard to do. We will meet in ETB on Friday evening and get it done!
Friday April 26,
And its DONE!!!! After several more hours in the ETB we are DOOONNNNNNNNNEEEEEE! we ended up not using the 3rd kinect as it would not connect to the system(for some unknown reason!!) JJ and I finished putting the video in the proper order that the girls requested and also we found an optimal positioning for the two kinnects such that the dancers are able to use the entire stage space and still be detected by the system. Here is a video of the practice run.
Monday April 29,
Well today was the day! We performed and thought it went quite well! Everything went smoothly after set up despite a few mishaps. Here is a video of the final performance, enjoy!
No comments:
Post a Comment