CURRENT PROJECTS


TRIDIMENSIONAL FACIAL RECONSTRUCTION USING POINT CLOUD AND POLYGONAL MESH TO 3D TELECONFERENCE

Example of face 3D model rendering: (a) method 1; (b) method 2.

This work presents an approach to reduce 3D information processing, provide viewpoint choosing and improve the quality of mesh generated by Convex Hull (CH) method. The CH method generates mesh without concavity information, so we have applied a triangulation by parts to reduce the lost of concavity information.

Point cloud of users’ faces were obtained and polygonal meshes were generated to apply them in teleconference or WEB free-viewpoint . We have made comparisons of performance of the construction mesh using traditional method and reconstruction by parts. We have concluded that, in this case, the concavity information was improved and the reconstruction time was fast.



Publications:

  • SILVA, J. P.; RIBEIRO, D. G. ; KURASHIMA, C. S. . Three-dimensional Facial Reconstruction Using Point Cloud and Polygon Mesh in 3D Teleconferencing. In: Symposium on Virtual and Augmented Reality (SVR) 2015, 2015, SAO PAULO. Symposium on Virtual and Augmented Reality (SVR) 2015, 2015.

  • SILVA, JOSIVAN; COSTA, GEIZA CARULINE ; MELO, JOÃO ; MASCARENHAS, IANN VICARI ; KURASHIMA, CELSO S. . Facial Modeling for Interactive 3D Reconstruction in Web Applications. In: the 20th Brazilian Symposium, 2014, João Pessoa. Proceedings of the 20th Brazilian Symposium on Multimedia and the Web - WebMedia '14, 2014. p. 203-206.

  • SILVA, J. P.; MELO, J. V. C. ; KURASHIMA, C. S. . Reconstrução 3D via HTTP. In: Workshop de Realidade Virtual e Aumentada, 2013, Jataí. X WORKSHOP DE REALIDADE VIRTUAL E AUMENTADA. JATAÍ: CCOMP - UFG - CAMPUS JATAÍ, 2013. v. 1. p. 215-219.





INTERACTIVE APPLICATIONS FOR DIGITAL TELEVISION

First Aid application running in Ginga midleware emulator.

Interactive applications are currently available in Brazil due to the implementation of the open digital terrestrial TV signal. This new technology might be used jointly with the applications installed on portable devices – working as a second screen –, that offers a differentiated experience to the user.

The objective of this project is to develop interactive applications for digital TV and a second screen application, but mainly our work consists on the analysis of how factors related to interface and protocols may influence user experience.


Publications:

  • COSTA, G. C.; KURASHIMA, CELSO S. . Second Screen Interactive Applications for ISDB-Tb Platform. In: 2015 IEEE Broadcast Symposium, 2015, Orlando, FL. Alternative Delivery Session IEEE BTS Symposium, 2015.

  • MELO, JOÃO ; MASCARENHAS, IANN VICARI ; COSTA, G. C. ; KURASHIMA, CELSO S. . Estratégia de Desenvolvimento de Software Interativo com Ginga-NCL para TV Digital. In: XLIII Congresso Brasileiro de Educação em Engenharia, 2015, São Bernardo do Campo. Sessão de Comunicações SC01 do COBENGE 2015, 2015. p. 1-10.







CHAT APPLICATION – DIGITAL TV

Chat application schema.

The project consists on the development of an interactive chat application for digital TV. Using a second screen application (on a tablet or mobile phone), users can share opinions and comments while watching television. The user texts through the Android application and the data is sent to a server. The message information is then acquired by the television software and displayed on the main screen to all viewers, along with a timestamp. As shown in the picture, the device sends a message (1) to the server using a PHP code, the message is then included to a XML on the server (2). The TV running its interactive application requires the XML file with all the massages to the webservices.

Interactive application are currently available in Brazil due to the implementation of the open digital terrestrial TV signal. This new technology might be used jointly with the applications installed on portable devices – working as a second screen –, that offers a differentiated experience to the user.

This project objective is to study and find relations between the use of interactive television and second screen application using a chat application to contextualize a social network on Brazilian ISDB-t system.




A SYSTEM FOR 3D TELECONFERENCE OVER IP NETWORK

Example of face 3D model rendering: (a) left side image 1; (b) right side image; (c) Depth Map Generated from two views; (d) Final 3D model with texture mapping.

This paper proposes a 3D real-time video-teleconferencing system over IP broadband networks. The video capturing uses two synchronized cameras positioned side by side in front of the person who will communicate, in order to compute his or her three dimensional information to be transmitted to the remote side.

The system also has a module for IP transmission of audio and video, and a module for 3D scene visualization. The implementation requires no specialized hardware, only personal computers with webcams, which simplify and reduce the equipment costs. As a result, 3D videoconferences were successfully realized inreal-time. Improvements in 3D mesh rendering and visualization interactivity are planned in future work.



Publications:

  • RIBEIRO, DOUGLAS GALETTI ; SILVA, JOSIVAN PEREIRA DA ; KURASHIMA, CELSO SETSUO . A System for 3D Teleconference over IP Network. In: 2015 XVII Symposium on Virtual and Augmented Reality (SVR), 2015, Sao Paulo. 2015 XVII Symposium on Virtual and Augmented Reality. p. 71.



REAL-TIME PEOPLE DETECTION AND TRACKING USING 3D DEPTH ESTIMATION


Comparative images among the detection: (a) Foreground blobs at 2.5 m; (b) Haar cascade classifiers for detection of frontal face at 2.1 m; and (c) Haar cascade classifiers for detection of upper body at 2.1 m.

Detecting and tracking people in video have a wide variety of applications in computer vision. However the development of robust methodologies for these purposes are challenging due to complexity of the scenes. This project proposes a real-time algorithm for people detection and tracking by tridimensional depth estimation in indoor environment. The approach is based on people detection and stereo processing techniques for 3D depth estimation of the analyzed scene. Detection methods investigated are based on different techniques: i. Foreground Blobs detection; ii. Histogram of Oriented Gradients - HOG, for human detection; iii. Haar Cascade Classifiers for detection of human frontal face; iv. Haar Cascade Classifiers for detection of upper body; and v. Haar Cascade Classifiers for detection of full body. The prototype implementation showed satisfactory results for people detection and tracking within pre-recorded video and also in real-time captured video by a stereo camera. The performance on standard hardware and using open source software library achieved a frame rate of up to 7 fps.