General fluff => Announcements => Topic started by: Goombert on February 02, 2019, 09:54:42 PM

Title: New Dependency on Google Protocol Buffers
Post by: Goombert on February 02, 2019, 09:54:42 PM
I wanted to make sure to cover this with an announcement to keep everyone in the loop. We have now rearranged the backend to the compiler so that it now uses Protocol Buffers directly. EnigmaStruct is now deprecated and should not be used for building new command line or frontend tools that integrate with ENIGMA.

We also had to come back and address a few regressions from this change in several games. With this, there have been no other new regressions discovered.

EnigmaStruct is deprecated rather than obsolete because it is still used by LateralGM. It will likely remain that way as my development attention is staying focused at RadialGM now which uses the new Protocol Buffers directly for its data model. One advantage to the new Protocol Buffers interface is that binary compatibility can be maintained when changing the protos. This is not true of the old EnigmaStruct, and thus, tools that use it will be subject to frequent breakage as a result of the lack of binary compatibility.

Long story short, you will now need to install Google Protocol Buffers when setting up ENIGMA or the next time you update an existing installation.
Code: (Bash) [Select]
# MSYS2 64-bit
pacboy -S protobuf:x
# MSYS2 32-bit
pacboy -S protobuf:i

I have also updated the Ubuntu/Linux installation instructions as well as the easy method script with the help of TKG. You will need a newer protobuf version which is why we recommend installing the dependency from Maarten Fonville's PPA which is where our Travis CI build obtains it.
Code: (Bash) [Select]
sudo add-apt-repository ppa:maarten-fonville/protobuf
sudo apt-get update
sudo apt-get install libprotobuf-dev protobuf-compiler

You will also need to do a clean build of the compiler.
Code: (Bash) [Select]
make clean
make -j4
Title: Re: New Dependency on Google Protocol Buffers
Post by: Goombert on February 12, 2019, 10:34:23 PM
Alright, a small update to this, we've finally completed the transition to Protocol Buffers over some additional pull requests and Proto2ES is no longer part of the codebase. This means the compiler uses protocol buffers directly which is handed to it by emake and RadialGM. Tools like LateralGM that use the old interface still pass an EnigmaStruct, but it is first converted into a protocol buffer before the compiler builds it.

I have not benchmarked, but theoretically, the protocol buffers and any frontend tools that use them, including emake, should now be reaching maximum build time performance. For the time being, tools like LateralGM will continue using the deprecated EnigmaStruct interface, subject to binary-compatibility breaking changes. My next step is to finally switch us to libpng and remove lodepng from our codebase before moving on to the next big things. This should bring even quicker build times!