Week 6

24 Nov

Another new week had started, the first thing to do when I reached the office is to the test the OSC to X11 application over and over again together with leonard. It was confirmed that it was the firewall that was stopping the packets from sending over to another computer. So we actually disabled the firewall before doing all the testing and the application seems to be working fine. 

The DV cam is also ready, so after Ming Hong and Wilson finished setting up lightdraw with the new DV cam, Leonard and I started to play around with the new DV cam with a little tutorial from Ming Hong on how to operate the camera.

Soon, we went over to the 2x3 Tile Display (orion) room to test lightdraw and this is the results we got:

• 1 Tile : The camera was able to capture the whole one tile, including the corners.
• 4 Tiles : The camera was able to capture the center of the tiles, some corners
• 6 Tiles : Some points in the center with almost none at the corners

Not forgetting to say that the calibration method and the angle where the laser is pointed does affect the results. We updated Kevin on our findings and problems found and he suggested that if we place the camera at the certain angle, will the camera be able to capture laser pointed at an angle, and we did tried that, and the results seems to be the same. the laser seems to refract through the tile having little or no laser detected.

25 Nov

We told Bernard that it was confirmed that it was the firewall that was blocking the packets from sending and was told instead of disabling the whole firewall, we will have to open the port for listening and while I was testing the application with only the port needed opened, Leonard went on testing the puzzle application for multi-user to ensure that everything went fine as we have to give a presentation to Kevin after lunch.

The presentation went out fine and we made a trip down to Fusion World to take a look at how lightdraw was setup there. 

We did some comparison with the setup there and the setup at the Tile Display and discuss the possible solutions with Kevin. Instead of capturing the whole 6 tile, we will be capturing only 2 x 1.5 of the tile. because applications such as VMD and paraview only need mouse movement in the left and center of the screen and so we tried using remote calibration to capture 2x1.5 of the screen and we found problems with the Makefile with doesn't allow the application to compile. Have Bernard to install Opencv into the remote desktop but it doesn't seem to help much. Ended the day with the application unable to compile.

26 Nov

Having problem with the Makefile, we had seek for Bernard's help. We were taught how to change the Makefile according to where the opencv file is installed. However, after trying for hours, there is still no progress. Had a trip down to the Sky Garden and level 14(where all the interns are) after lunch and we were really glad that we are at Level 16 doing lightdraw.

After that, we continue changing the Makefile but no progress was made. This problem was kinda serious as both Kevin and Ming Hong came and help and they just throw whatever that the system wanted and finally the application was able to compile and run smoothly and so Bernard went on opening the port for us to use but it doesn't seem to be working and the head node went 'crazy' whenever we did something to the firewall. At the end of the day, Bernard told us to use any port that is opened inside iptables. 

27 Nov

Almost all the staffs went for the retreat at level 15 leaving us alone in the lab. Since there's a demo at the tile display room in the morning, we were told to 'touch' the tile display only after lunch. Soon after lunch, we checked the iptables and started using the 'available' port via trial and error. We ended up using port 53, and started to test the remote calibration but there seems to be problem with it and so we stick to manual calibration and the OSC to X11 application seems to be working fine its just that lightdraw seems to be taking the coordinate of the macbook instead of the screen.

28 Nov

We tested lightdraw and remote calibration using the 2 macbooks and other than the problems we found earlier on with the remote calibration, Ming Hong also found that there is some problem with the lightdraw codes. After Ming Hong changed the codes, we started testing again and we came out with this conclusion:

• The laser must be pointed straight, not at an angle

Reflection:

This week, I learned how remote desktops can be used to communicate with each other. Just when 2 concurrent users are using the same application, there can be used to communicate with each other via the terminal. I also learned some simple scripting on bash such as how you can give access to users. I have also found out that we can even use functions such as if-else etc simply using the terminal without writing a .cpp/.c class. 

Week 5

17 Nov

Continued with the snapping function of the puzzle application for multi-user since I stepped into the lab in the morning. Leonard joined in when he arrived and we started testing the application. Things were working fine and we also had fixed some bugs whereby rotating piece did not focus and 'pop up' to the front when there's another piece overlapping the rotating piece.

Soon after, we took one of the MacBook Pro and webcam to the 2x3 tile display (orion) room. When went around looking for a suitable location for the placement of the webcam and soon, we decided to place the webcam infront of the screen because the frame at the back was blocking the webcam making it difficult or impossible to capture the whole screen.

And this is our findings/problems when placing the webcam infront:

• The screen is too bright, when the laser shines on the screen, the screen seems to be brighter than the laser
• The webcam is too small and the quality is not sharp enough to capture the laser
• The laser seem to pass through the screen instead of reflecting it

18 Nov

Since the DV camera is not here yet, we decide to start coding a simple application which listens to the OSC packets and control the X11 on the client computer. 

19 Nov

Had a team meeting with Bernard in the morning to discuss about the problems and possible solutions of setting up lightdraw on the 2x3 tile display. Had played with the SmartBoard in the room before the meeting had started and got to know briefly how it works and soon we came out with the task and objectives to be done for the whole week. 

Today, we will be setting up lightdraw on one of the node of the 2x3 tile display (n22) as suggested by Harold and see if it works well. We will be testing by placing the webcam both in front and at the rear.

And this is our findings

• Placing the webcam at the rear is almost impossible even though it was hanged above on the frame because the light from the projector was way too bright that it affect the lightdraw application from working normally
• Placing the webcam infront of the screen, hanging on the ceiling is the only possible way because it doesn't block the audience view and the surrounding environment enable lightdraw to work normally.
• The laser can only be detected only in the center of the node but hardly detected at the sides. It was suspected that it was because of manual calibration and the quality of the webcam that causes it.

20 Nov

Started the day by continuing with the application to be used on the client's computer. Tried for almost half a day and found there will lots of libraries missing in the make file. Edited the make file but was still unable to compile. Had switched focus to the puzzle application again because there will be kids of the staffs visiting tomorrow as told by Cheng Teng and from previous experience, we decide to test the application over and over again. As expected, we found bugs again. Its actually the same bug when you try to drag the puzzle pieces out of the window but this time round, it was the snapping that causes the crash. Below is an illustration drawn by Leonard:

If the user tries to snap at the extreme corner with sufficient for snapping to occur, the snapped piece will move out of the window and cause the whole application to crash and we managed to solve the problem before we left the office at the end of the day.

21 Nov

Today is the day where the kids will come and play with our lightdraw application and so I asked Leonard if he could do the final testing of our application while I continued doing the application that controls the X11 of the client's computer. 

Soon I was told by Leonard that he and Ming Hong had found another bug that is when a user is rotating and the other user was snapping, the snapping will not occur until the rotation had ended. Lucky we did a final testing and solved it before the actual event started and just minutes before the event started, I managed to control the mouse on the MacBook Pro using OSC. This are some pictures taken during the event:



After the event, I've tried control the mouse on leonard's macbook but it doesn't seem to be working, it was suspected that a firewall was blocking the packets from sending to other PC as told by Ming Hong. Will try to solve the problem on Monday.

Reflection:

This week, I managed to see a clearer view on the importance of getting prepared for anything. As with test our applications again and again, we found more and more bugs and tried to solve it. The more you run through the application and finally you found no bugs, you will have a great boost in confidence level because you know that everything will run smoothly when a presentation or event started. It is really important to think in a user point of view so as to give them what they expected and exceed their expectations.

It is also important to set a target everyday or every week so you will be able to strive hard for the target and get it done by the required deadline. Otherwise, things will just keep on hanging at the same position if there isn't any motivation at all.

Week 4

10 Nov

Another week had started, and we are still having trouble make the puzzle application multi-user and so we went on integrating the application to 'adapt' to the new lightdraw codes that Ming Hong had edited and that took us almost a day to do it as we had to change quite alot of codes hence little time for making it multi-user.

11 Nov

Ming Hong is on leave today so we really had a hard time trying to implement multi-user. First of all, we had difficulties understanding the function that Ming Hong had written though it already had comments that shows what each code does but no ones understands it. When Kevin came for an update we had briefly explain our current problems and the possible solutions which at the end of the day, we realised that it is not workable. Kevin also briefed us on what we will be doing for the next few months. Ryan and Wilson will be working on lightdraw applications using quartz composer whereas for Leonard and I will be working on the 2 x 3 tile display (orion). He also demoed to us what the quartz composer can do and serious the graphics is really cool.

12 Nov

Things wasn't going well today. The multi-user doesn't seem to be working though Ming Hong had explained how his function works. As for the other group, he successfully integrated multi-user but is was only for 2 paddles. Had tried quite a number of ways to integrate it but had no results.

13 Nov

Kevin gave us more details on what Leonard and I will be working on. Basically, we had to integrate lightdraw to work on the tile display. So told by Kevin, we will be using writing an application to make the remote desktop to listen to the OSC packets from lightdraw to control the X11. Hence, we had to decide on the placement of the camera, ensure that OSC could control the X11 and also integrate lightdraw into the 3d application.

After lunch, Bernard, the on in-charge of the 2x3 display brought us up to the server room to take a look that the architecture of the servers and we had a lesson on how it works. Soon after, we had continued on multi-user for the puzzle app and after much struggle we finally could display the rotation corners when "mouse-over" but we couldn't rotate and drag and not to say snap it and the deadline is tomorrow!! 

14 Nov

Kevin came in the morning and asked for a presentation from the 2 groups. We told him that the multi-user wasn't working well and so he told us to test the application that only run single-user. The presentation doesn't turn out good. Kevin, Harold and Bernard commented on our applications. We had also learn more about LightDraw's ease of use and the visual effects of the applications. 
Above is a picture of Kevin testing our application. As soon as we learned our lesson, we went on editing our codes and we found that it was the waitKey the dragging to be "blocky" and the offset causing the inaccuracy of the rotation box. Before lunch, we managed to change the code and update Kevin about it.

During lunch, we were told that we had to give another presentation to Terence and thanks to the lesson learned this morning, the presentation appear to be much better. After the demo, with the help of Ming Hong, we managed to drag and rotated the puzzle for multi-user.

Reflection:

It has been a month since we joined IHPC as interns, this week, I have learned that the new lightdraw code is written in the way to make it work more effectively. I have also learned more about DMX and chromium, their pros and cons and server architecture. I learned the differences between the presentations in school and the presentations at work. Though we had lots of presentations in school, it is different from presentation at work, we not only have to understand we have done inside out but also be prepared for every presentations so as to show professionalism and respect not only to the people listening to the presentation but also to ourself. 

Week 3

03 Nov

A new week and some good news. We finally managed to snap the sub-images together!! And soon, we had integrate rotation into the application. Today, the whole IHPC had some seminar to attend so we can hardly see anyone in the office.

Had given Kevin an upadte before lunch and I guess it went smooth. Lunch was special today because we had Crystal Jade. It was expensive of cause lucky we just gotten our pay and it's a 'once in a while' thing, so I guess it's alright. So at the end of the day, this is how our application looked like.

04 Nov

Ming Hong and Kevin had again went for a seminar and we are left in the lab on our own for almost the whole day. Did not really do much coding today, just had changed the inital array of array to a vector of vectors so as to store the position of the sub-images more easily as compared to an array. With a vector of vectors, the index of the sub-images would tell us the orginal position of the images and so made the algorithms looked less complex.

05 Nov

The puzzle game was somehow consider completed and so we tested it with lightdraw using X11. We had before that add in the winning message at the end of the game so that it would looked more completed. 

Testing on lightdraw looks successful and so we went on trying to integrate Open Sound Control but know real little about that so we went on doing research and asking Ming Hong about it. Had change the makefile just to integrate OSC into the application. Being stucked and really bored trying to solve the integration problems, we went to play the black board that Ming Hong created and this is what we drew. Cool huh?

Not forgetting the other team that is also working hard on their apps. They able to check for collision and make the ball bounce more naturally. 

06 Nov

After some understanding from Ming Hong on how the OSC actually work, we actually started editing our codes so that it would take the coordinates of the laser instead of the mouse. It was not as easy as we initially thought because OSC does not have separate event such as mouse up, mouse down with a normal mouse would have, so we had to think of alternatives to allow the laser to work as similar as the mouse. 

We had a hard some trying to edit the codes as it become kind of dis-organized when all the mouse events are together in the same loop. However at the end of the day, we managed to display the squares for rotation and rotate it.

07 Nov

Since we had managed to rotate and display the squares, we went on editing the codes for dragging and snapping. By lunch time, we only managed to drag the images but not snap it. Until after lunch we found that it was because the machine did not 'go through' the snapping code.

Today was special too because we had our lunch at VivoCity because Kevin had to settle his visa.

And finally, at the end of the day, we had completed integrating the application with OSC. However, this application is only for single-user. Let hope we are able to allow multi-user by next week. Here is a video showcase of the application. 

Reflection of the week:

This week, we learn more about Open Sound Control(OSC) that is used for the integration of lightdraw. Basically the application listens to the port that sends packets containing the coordinates of the laser where it points so that it could replace the mouse condinates. The problem that we encounter is that a laser had limited gestures unlike the mouse which have a left and right button. Also, OSC does not have separated event like the mouse up, mouse down, mouse click event which the mouse has. Hence we have to find ways to make the laser work as if it was like a normal mouse event.

Another thing is that we are more used to coding using text editor thanks to the everyday 'training' that we had in our lab. That is a skill that students in this generation lacks of because of the over dependent of IDEs.

Week 2

27 Oct

A break from work because it's a public holiday.

28 Oct

Back to work at IHPC and we shared our ideas to integrate with lightdraw when Kevin came for an update in the morning. So after discussion, we decided 4 games:

• 4 Player Pong
• Memory Game
• Scrabble
• Sudoku

However, after lunch Kevin had a change of plan. Instead of the 4 games that we discussed, we will be doing 2 - a puzzle game and 4 player pong because it would require larger screen as that was why lightdraw was initially developed for.

So we were divided into groups, Wilson and Ryan will be developing the 4-player pong game and Leonard and I will be doing the puzzle game. And so leonard and I tried to find some open-source codes online but there isn't any results until Kevin came and told us that it would be easier to write our own code as we may not be able to easily understand the codes found on the web.

Sometime in the mid afternoon, we had a course on C++ taught by Ming Hong and Cheng Teng and at the end of the day, this is what we have done.

29 Oct

We finally started on the development of the puzzle game. We had learn how to set the an image area of interest today so as to crop the images into pieces so that it would look like a PUZZLE!! At the end of the day, we had manage to load, crop and show all sub-images on the window. Not to forget the other team doing the pong game. They had their UI out!! can't wait to play these applications on lightdraw!!Great Start!

30 Oct

Today, we had manage to randomly place the sub-images on the screen and I also managed to allow users to define the image path and the number of pieces that they are comfortable with. Thanks to Ming Hong, that was great help.

31 Oct

When we were all busy working, its already Friday!! hmm.. today we had manage to randomly rotate the puzzle when it first start. Also, we had manage to resize the image of the puzzle regardless of the image size defined by the user.  Most importantly, we had managed to drag the pieces at the end of the day and The "Pong" team isn't losing out either, they had managed to move the paddle using key events and able to check collision. Well, really had learn alot from the people at IHPC. I can only say we are like a baby learning how to walk.

Reflection

This week, I would say it was more on hands-on and communication skills. Had a one-day intensive C++ and OpenCV course. Communication skills had always been a problem for me as my english had always been poor. But this will not stop me from learning and improving. I can see that Kevin is trying to help me alot in that and I'm also trying hard to improve. Also, we had learn how to use openCV functions such as 2D/3D array, resize image, image rotation, taking in user input from command prompt, random, setting image area of interest and more. Functions such as 2D/3D arrays are what we seldom use when doing programming in school but was commonly used in the industry. We also learn how to write a GUI application without the use of any IDEs. From this, we actually realised that we depend too much on IDEs when doing programming with GUI. Thanks to the fundamentals that the school had provide us that allow us to understand the complex algorithms in the shortest time.