top of page

[CEDEC+KYUSHU2024] A detailed explanation of time-shifted streaming

中島 健太郎

Updated: Dec 17, 2024

Due to the one-hour time limit, we were only able to explain a part of our two major upgraded projects, RIGDOCKS and SKYSQUARE.

There are a number of elements that I would like to explain in more detail, and I hope to provide additional explanations in this blog.

If you watch the time-shifted broadcast and read this article to understand the details, you will be able to understand the content in more detail.

*The distribution period will be two weeks from today until December 11th (Wednesday) .

Speakers
 

November 23 (Sat) 14:00-15:00 Speakers: Kentaro Nakajima, Sarina Takeda



[Lecture Content]

RIGDOCKS is a set of APIs required for creating scripts for Repaer. In order to enable sound creators to create their own scripts, we have prepared Japanese documentation and implemented many functions to meet the needs of the field. In this session, we will focus on the improvements made to the BRONZE/SILVER/GOLD versions of RIGDOCKS included with CEDEC2024.

We are also working towards the realization of a large-scale field studio called "SKYSQUARE." The realization of Japan's first field studio is currently facing a very difficult road. We will report in real time on how we are solving these challenges and the current situation.

What is needed to enable the recording of various sounds that cannot be achieved with modern Foley recording? How do we solve these problems? In this session, we will announce how many of these issues are currently being solved.

 

table of contents

 

◇Lecture begins

cover

1: Significant cost reductions with cutting-edge automation technology

The current state of "RIGODCKS//SKYSQUARE" that will lead to Japan's first creative endeavors!!

 

In order to inform you of the current status of our two major projects, which are constantly growing,

I made it the title.

RIGDOCKS has updated two days before CEDEC+KYUSHU will be held on November 23rd.

We wanted to include that information as well as share as much as possible the new features and updates we've just added.



2: Company introduction, speaker introduction, official mascot introduction

 

We gave a company explanation, introduced the speakers, and introduced our official mascot.

My main job is developing game sounds, and I am also working on two major projects, RIGDOCKS and SKYSQUARE. After my main job of developing game sounds, I want to be able to give as many people as possible a taste of what is wanted on the ground.

We launched the project.

We also have an artist support department called ALGAZODIK, where Takeda is a DJ artist. Loopdirock is our official character. Anything modeled can be executed as a motion, and is used in various advertisements for our booth, including promotional videos.


CEDEC2024 in Summer

3: You can watch the CEDEC2024 presentations on YouTube

 

Due to time constraints, I will explain things quickly.

Please check out this YouTube video for details about the birth of RIGDOCK and our plans. You can also compare how it is growing.

Although CEDEC2024 will take place in August and November with only a three-month difference, we are currently moving forward at full speed.


4: The latest PV of the two major projects RIGDOCKS//SKYSQUARE

 

This PV is exclusive to CEDEC+KYUSHU2024.

Loopdirock also appears in this PV which expresses the challenge of RIGDOCKS//SKYSQUARE. It is an image video which combines cutting-edge digital technology with the rugged analog expression of Japan's first and largest Foley recording.



5: The birth of RIGDOCKS – Five issues

 

Why was RIGDOCKS born?

This was born out of five challenges.

The issues we are facing are problems that arise in various game sound fields.

I came up with my own unique solution to this.

This issue is not unique to our company, but is caused by the expansion of game development this year.

In light of modern issues, we aim to improve the field of sound development, which is a specialized profession.

The aim is to encourage more creativity.


6. What is RIGDOCKS? -RIGDOCKS-BRIDGE-

 

RIGDOCKS is a set of APIs that can be executed using the LUA/PYTHON scripting language.

RIGDOCKS-BRIGE- has a license role and a common function role.

The license works in conjunction with the web and BRIGE.

The current MultiUserLicense features will be automatically reflected in one account purchase.

It can be used by up to 5 people with one purchase.


BRONZE UPDATE

7. What is RIGDOCKS? - An extension of BRONZE

 

In addition to bug fixes, array handling has been expanded. Some examples are shown exclusively in this blog.

Please see the release notes here for information on changes in this version.


Example: Putting selected media into an array

 

▼Before - Get the number of selections and register the media in an array using a For statement -

 

dofile(reaper.AZ_GetLuaInitPath())

require("reaper_AZSTOKE_BRONZE")


allCount = reaper.AZ_GetMediaTypeCount(0,1)

itemList = {}


for i = 0,allCount-1 do

itemList[i+1] = reaper.AZ_GetMediaItemSelect(0,i,1)

end

 

▼After - Output the selected media array of the specified project -

 

dofile(reaper.AZ_GetLuaInitPath())

require("reaper_AZSTOKE_BRONZE")


itemList = reaper.AZ_GetSelectedMediaItemList(0)

 

In this way, we have succeeded in incorporating the settings for obtaining arrays of selected media, which is often used, into the API. Since it can even output the array, each code has been minimized.

It is possible to execute scripts at higher speeds.


SILVER UPDATE

8. What is RIGDOCKS? - An extension of SILVER

 

We are mainly working on fixing bugs. Please check the release notes here for details on the bugs.


SILVER comes with the patented technologies "HANDAUTOMER" and "TALKTRON".

It has automatic voice adjustment and automatic control in multiple languages.

It consists of mid-level APIs.


GOLD Update

9. What is RIGDOCKS? - An extension of GOLD

 

This is Plan GOLD, which has been significantly expanded from CEDEC2024.

It allows for the manipulation of Excel to add APIs that allow further tweaking of Wwise.

In this presentation, we will be demonstrating how to make full use of the expanded features of Wwise.

Please check the release notes for details of updates.

Check out the demo to see an example of using the Wwise extension.


10: What is RIGDOCKS? -About the three patents used in the demonstration-

 

This is the patented technology that we also mentioned at the previous CEDEC 2024, and this time we explained three patents that will be used in the demonstration.

For detailed information on each patent technology, please see the links below.


We currently hold four patents and have our own cutting edge automation technology.



11: RIGDOCKS demonstration

 

This demonstration is automated using the Wwise extension that uses GOLD 1.1.1.

Adjust the files in the folder and run the wav waveform from Reaper to Wwise fully automatically.

By provisionally adjusting the data and implementing it,

This will reduce a lot of post-recording work.

ReaScript increases the possibility of automating various tasks that were previously done manually due to the complex procedures required for implementation.


The ReaScript used in the presentation uses Lua. Please check the Reapdock here for details.

 

dofile(reaper.AZ_GetLuaInitPath())


require("reaper_AZSTOKE_BRONZE")

require("reaper_AZSTOKE_SILVER")

require("reaper_AZSTOKE_GOLD")

RIGDOCKSのAPIを利用可能にする

cancelType, inputList = reaper.AZ_GetUserInputList("Insert",1,"Path:","D://")

フォルダの入力をユーザーに行う機能を持ちます。
納品データのフォルダパスを指定します。

if cancelType then

テキストボックスでOKを押した場合は以下を実行

--file import HANDAUTOMER

fileList = reaper.AZ_GetFilePathList(inputList[1],"wav")

フォルダパスの中に入っているwavデータをすべて配列に置き換える対応 

startPos = 0

最初にインポートする波形の位置を設定(秒)

itemList = {}

保存用のアイテムリストの空の配列を定義  

for i, value in pairs(fileList) do

wavデータの配列fileListのForループ

itemList[i] = {}

itemListの指定番号内さらに配列を定義する

itemList[i]["item"] = reaper.AZ_InsertMediaTrackIdSecond(0,0,value,0,startPos)

開始位置とfileListのパス情報を得てトラック0番にメディアをインサートする。
そしてitemListにitemを配列として入れていく

reaper.AZ_SetMediaItemHANDAUTOMER(itemList[i].item,3)

インサートしたメディアアイテムにHANDAUTOMERを実行 圧縮率は3

itemList[i]["name"] = reaper.AZ_GetMediaItemName(itemList[i].item)

       インサートしたメディアアイテムの名前を取得

--get parent

itemList[i]["parent"] = ""

splitList = reaper.AZ_GetStringSplitList(itemList[i].name,"_")

 メディアの名前の[_]を使って分割したものを配列に登録

table.remove(splitList)

分割したものの配列の最後を削除する

for e, value in pairs(splitList) do

if e == 1 then

itemList[i]["parent"] = itemList[i].parent..value

else

itemList[i]["parent"] = itemList[i].parent.."_"..value

end

end

再度[_]でつなぎ直すことで pl0001_vo_ACTION_01 → pl0001_vo_ACTIONになる

--next start pos setting + regionMarker

endPos = reaper.AZ_GetMediaItemEndTime(itemList[i].item)

メディアアイテムから終了地点(秒)を取得

reaper.AZ_AddRegionMarker(0,startPos,endPos,itemList[i].name,0,0)

開始地点(startPos)と終了地点(endPos)とメディアの名前を利用してリージョンマーカー設定

startPos = endPos + 1

次のメディアをインサートさせるために終了地点からプラス1秒後にstartPosを設定

end

--render setting

renderPath = "D:\\script\\renderdata"

renderFileName = "render"

レンダーのパス情報の取得準備をします。
レンダーが保存されているパス情報と名前を設定

jsonStr = reaper.AZ_ReadFile(renderPath,renderFileName..".txt")

指定のパスと名前からテキストデータを取得して文字列に変換

render = reaper.AZ_Json_Deserialize(jsonStr)

文字列データから配列に切り替えてrenderを生成

path = reaper.AZ_SetProjectPathFolder("Rec",3)

プロジェクトパスにフォルダを生成し日時がフォルダに追加されることで同名フォルダを防ぎます。

render.OutputDir = path

 レンダーのアウトプットするパス情報のみを書き換えます。

--render

fileList = reaper.AZ_RenderToAudioFile(0,render)

レンダー情報に合わせて指定ファイルをレンダーします。   

--wwise intro

wwisePath = "\\Actor-Mixer Hierarchy\\Default Work Unit\\Master\\VOICE"

parent = ""

Wwiseの初期設定を行います。WwisePathは、パス以下にボイスをいれるための設定です。

if (reaper.AZ_Wwise_Connect("127.0.0.1",8080)) then

    Wwiseにコネクトします。   

--wwise object create

for i, value in pairs(fileList) do

   レンダーで書き出したファイル(fileList)分Forループ対応       

if parent ~= itemList[i].parent then

parent = itemList[i].parent

containerList = reaper.AZ_Wwise_CreateContainer(wwisePath,0,itemList[i].parent)

parentを利用してコンテナの生成(ランダムコンテナ)を行います。

end

soundVoiceList = reaper.AZ_Wwise_CreateSoundVoice(containerList.ID,value,itemList[i].name)

itemList[i]["soundVoiceID"] = soundVoiceList.ID

生成したランダムコンテナの中にSoundVoiceを生成します。
valueにファイルパスが設定されているので同時にインポートも実行できています。
そしてitemListにSoundVoiceのGUIDを登録しておきます。
これで必要なWwiseのファイルを生成します。

end

--LOUDMATCHIFY Savedata

jsonStr = reaper.AZ_ReadFile("D:\\script\\savedata\\savedata","savedata.txt")

loudmatchifyList = reaper.AZ_Json_Deserialize(jsonStr)

Loudmatchifyのセーブデータを取得して配列に戻します。
※Renderのセーブデータと同様の方法になります。 

--loudness

for i, value in pairs(itemList) do

itemList分Forループ対応

wwiseVol = reaper.AZ_MIDDLESCANNER_Wwise_GetVolumeForMediaItem(itemList[i].item,"English(US)")

MIDDLESCANERを実行して音量変化量をWwiseから取得します。

reaper.AZ_LOUDMATCHIFY_SetVolumeForMediaItem(itemList[i].item,loudmatchifyList,1,wwiseVol.Volume)

Wwise変化量と部分一致のセーブデータを基にラウドネス調整を行います。

Msg(itemList[i].name)

Msg(wwiseVol.Volume)

Msg(reaper.AZ_GetMediaItemLoudnessMaxMomentary(itemList[i].item))

Msgはデバック用になるのでコンソールに出力させて状況を確認できるようにしています。

end

path = reaper.AZ_SetProjectPathFolder("Rec",3)

再度プロジェクトパスにフォルダを生成

render.OutputDir = path

レンダーに生成したパスに設定

fileList = reaper.AZ_RenderToAudioFile(0,render)

レンダーを実行

for i, value in pairs(fileList) do

reaper.AZ_Wwise_ImportAudioFile(itemList[i].soundVoiceID,value,"English(US)")

end

再度Wwiseにインポートを実行して入れ替える

reaper.AZ_Wwise_Disconnect()

Wwiseのコネクトを外す対応

end

end



SkySquare

12. SKYSQUARE

 

We will report on the latest happenings at SKYSQUARE.

We haven't yet gotten to the point where we can start recording.

We are in a situation where we are always taking three steps forward and two steps back, so we still have a long way to go.

I hope I can inform you of this situation.


13: Introduction to Field Studio SKYSQUARE

 

I introduced you to the type of land it is.

The total land area is 2,000 tsubo (6,660 m2) with a flat space of over 2300 m2.

For large-scale Foley recording, we prepare a space that can be recorded within this flat space.

There is a wide variety of land types, and the land below has a river running through it, making it ideal for recording water sounds.


14: You need qualifications to make it happen

 

In order to make this large-scale recording possible, we needed all kinds of construction-related licenses, and we obtained them.

To date, we have successfully obtained eight licenses, enabling us to offer a variety of services, including compatible models.

Construction industry licenses are highly specialized, and even transporting cargo requires various licenses.

Currently, our company's Takeda has also acquired the slinging certification, and we are now able to carry out the work with a two-person team.



15. You need construction machinery to make it happen

 

The photos and videos introduced at the previous CEDEC 2024 used a 3-ton excavator.

There were a lot of things that had to happen before I could borrow it.

It was my first time to go to a rental shop specializing in construction equipment, so I decided in advance what equipment I wanted to rent and the date.

It was done.

What happened when I borrowed it and what issues came up

Thinking that the quickest solution would be to buy a new excavator, we decided to purchase one.



16: To make it happen, we need materials

 

There is nothing that can be easily obtained at a home improvement store.

Large components of various materials will be required.

There will need to be a number of different methods in place to secure these materials.

I finally managed to secure a scrapped vehicle.

In order to obtain the scrapped vehicle itself, we asked a private mechanic who is an acquaintance of our staff to help us.

We purchase a car that is on the verge of being scrapped due to a malfunction of the electronic system, and transport it to the site.

We secured it after going through the vehicle scrapping procedures.


Once we had secured the scrapped vehicle, the first thing we did was test lift it.


In order to further secure timber and carry out felling,

I learned this from a forestry professional.

We conducted a training session using the equipment that will actually be used on-site, so we used that.

I would like to carry out tree cutting.




17: SPECIAL SALE

 

We are holding a special sale this time as well.


▼For the general public

We have a 20% off code available for a limited time.

If you watch the online session, the code will be displayed.

Please refer to this if you are considering purchasing.


▼Academic version (for students/teachers)

It is possible to issue an academic version for a limited time.

Please fill in the necessary information in the contact form and issue a special code to purchase.

Please check the image above for the details.


If you are considering using it, there is a trial version available.

Please consider using them.




6 views0 comments

Comments


bottom of page