Developing Interactive Educational Resources

-This page is aimed at digital material content developers.-

At ICER, we are supporting the development of interactive learning materials.
Materials for e-Learning (description document data, still or moving image data, 3D CG data) in general, need to be supplied by the faculty carrying out the lecture. With this data as the base, we can support you with know-how and skills to produce materials such as those shown below in demos.
If for example, you want students to have access to lecture contents via web browsers, so using iPad or iPhone they can access the content during lectures, or if you want to turn your educational material into an e-book which can be used out-with the university. Then please get in touch with the center.

  1. Development of interactive educational materials using 3D CG.
  2. Use of 3D CG is one very good method to increase students understanding of a topic. For example, in the medical field, a detailed 3D CG interactive, representation of a human body will allow students to focus on a specific area and zoom-in/out), enabling students to further study parts they have a particular interest in. Up till now, the limitations of still images has meant this type of self-study has not been possible.

    One method for creating 3D CG educational materials is shown below.

    1. Use of 'iBooks Author'
    2. There is one eBook standard called EPUB(Electronic PUBlication). The American eBook Standards organization the IDPF (International Digital Publishing Forum) is promoting this eBook file format. The newest version EPUB 3.0 was established in 2011 October 10th. This includes a facility for Japanese vertical type setting.
      Software used to view the EPUB file format is called 'EPUB reader'. For editing of EPUB files we can use software called 'EPUB Editor'. iPad and iPod EPUB reader software supports EPUB 2.0 files. Actually, the iPad EPUB reader application 'iBooks' can open not only EPUB 2.0 files but also its own iBooks format.
      iBooks format, was created by Apple, based on the EPUB format. Standard EPUB readers do not support 3D CG format or interactive use, whereas the iPad iBooks reader does.
      Using display of 3D CG data, the user can change the perspective from which they view. To edit files in the iBooks format, the 'iBooks Author' software is required. This can only be run on Mac PC(only Lion OS) and above. The iBooks Author software can be downloaded from Apple store.

      Below is an image of iBooks being developed on 'iBooks Author'. The iBooks contents can be downloaded by clicking on the image. If you install the iBooks application (free) to an iPad, then download the iBook content from a web browser, and you will be able to open it in iBooks.


      iBooks教材イメージ iBooks教材イメージiBooks教材イメージ iBooks教材イメージ

    3. Display of 3D contents by image transfer (under development)
    4. In developing an application to display 3D CG, it is necesary to use a specific set of graphics functions. This group of functions is called a graphics library; one of the most widely used graphics libraries is OpenGL. Web browsers up till now, have not had native support for OpenGL.
      Till now for web browsers to display 3D content they have had to use the PaperVision3d library to display Flash contents, or the Java3D library for Java content development. It was necessary to display 3D contents through the Flash Player plugin or Java Virtual Machine.
      Because of this, products that didn't support either Flash or Java3D would not be able to display 3D content. One method to solve this problem, was creating the 3D content data on the web server, and sending this data to the users web browser.
      At ICER we are currently developing such a web service system. In this system, the 3D contents data is stored on the web server, corresponding web contents are read into the web browser, and display of 3D content on the browser is possible. By using JavaScript, zooming in and out and changing the angle of view of the 3D contents on the web browser is possible.
      In this situation, the Flash Player plugin or Java Virtual Machine functionality is not required, so regardless of the web browser platform you are using, (iOS, Android OS, Windows PC, Mac PC) display of 3D contents is possible.
      However, while using this you need to be attached to the internet. Further, as the 3D images are being transferred from web server to web browser, bandwidth use is a problem. We are currently working on developing a system, which can reduce this heavy use of bandwidth.

      Below are a male skeleton data 3D CG representation and a brain data 3D CG representation.

      Aist Skeleton Model Anatomy Brain ModelSkeleton data from he National Institute of Advanced Industrial Science and Technology (AIST) (中村仁彦・山根克・諏訪元・近藤修・河内まき子・川地克明・持丸正明、2008:成人男性骨格形状データ(産総研H20PRO-905))

      This can be viewed on various web browsers, however Internet Explorer will display an error.

    5. Use of WebGL
    6. In the new HTML specification 'HTML5', the OpenGL Library has now got native support. This is called WebGL. For display of 3D CG contents, the structure data can be stored on the web server and then when the web browser receives that data it can display it. Using the web programming script language, JavaScript, zooming in and out and changing the angle of view of the 3D contents on the web browser is possible.
      Displaying contents in this manner, the Flash Player plugin or Java Virtual Machine are not required. So, if you are using a web browser which supports HTML5, regardless of the platform (iOS, Android OS, Windows PC, Mac PC) display of 3D content is possible. No special application software is required. However, you must be connected to the internet.

      Below is a Human Anatomy Atlas "BioDigital Human" represented in 3D CG using WebGL.

      BioDigital Human

      This can be viewed using browsers that support HTML5 WebGL, such as Google Chrome.

  3. Development of interactive educational material using VR, AR, MR technologies.
  4. With the creation of high immersive VR (Virtual Reality) devices, we have reached a time when systems using VR can be put to practical application. Also, with the emergence of small portable devices such as the iPhone, with its camera, GPS, Acceleration center functionality, specific types of VR: AR (Augmented Reality) and MR (Mixed Reality) can also be put to practical use. Putting these types of systems into use in education, is one effective method of improving quality and increasing students desire to participate in learning.

    Still at the development stage, however, we are developing educational materials as below.

    1. Examples of Educational materials created using Virtual Reality.
    2. Below is video of a pointing device called the 'Phantom', which uses force feedback, being used in a surgery training system. The surgery area can be extremely complex and varied so putting this system into practice is difficult. However, a VR training system like this is very useful. Representing the surgery area using 3D CG, and using the Phantom to model the operation, calculation of the incisions and stitches required can be done virtually. Use in lectures and labs for teaching the surgery process to students will increase their knowledge and awareness of surgical procedure. Using Phantom, students can get used to the feeling of applying a scalpel to a patient, and, if correct data is entered beforehand, the student can be warned if they attempt to cut the wrong areas.

      手術トレーニングシステム

      Demo movie

      Below is a movie of the Phantom being used in a dental training system. The Phantom models a drill, and the student can get a virtual experience of drilling a patients tooth. Again, if the correct settings are input beforehand, if the student drills the wrong tooth, warnings can be produced. If used in dental treatment lectures or labs, the system will increase student knowledge and awareness of dental surgical procedure.

      歯科治療トレーニングシステム

      Demo movie

    3. Example use of AR and MR in educational material.
    4. Below is, an image of MR being used in educational material. Using the camera of an iPhone or iPad we take a picture of an actual person, and superimposed over that person is a 3D CG bone structure. This is one use of AR and MR being used in the field of medicine. However, using the camera to take a background picture, we could for example, superimpose the image of a person from the past. In this case this the technology could easily be applied to History or Humanities.

      MR Appli No1 MR Appli No2