Add VideoPlayer Extension to All Platforms

Reporter: time-killer-games  |  Status: open  |  Last Modified: November 20, 2020, 11:23:21 PM

Win32 games need these deps bundled with their games via an installer or self extractor:
https://drive.google.com/file/d/1f-CL3z-PufjiHQDJvPwUkDLG1teZA3iZ/view?usp=sharing

MacOSX games need all of these deps in their app bundle's "/Contents/Frameworks" folder:
https://drive.google.com/file/d/1HJokueATBYzisammKECHPiD9cDde63AS/view?usp=sharing

Linux and FreeBSD users may simply use their package manager as usual to install libmpv.
Android support is also possible since libmpv has an Android port, so that's all platforms.

fundies  
I don't think we should be adding an extensions to launch specific programs. I think if we want a video extension it should be rendering to a surface directly and not launching some video player. If you want a general extension to launch programs I'm fine with that. You could then provide all these video_launch functions as scripts in a game instead. That way when you want to launch a dialog, or a video player or whatever it all uses the same code.
time-killer-games  

@fundies Josh said he doesn't want there to be an extension which does stuff with application process id's exposed to the user namespace and that I should host it on my own repository if people really want process stuff. My insecure_bullshit.cpp's would go better in the engine for internal use instead. It still won't be exposed to enigma_user. That way if multiple extensions need these features they'll have access to it in a place where it makes more sense than just in my video player extension lol.

The video embeds in the game window, so at least it's similar to a surface in that it doesn't open in its own window. Similar to zenity/kdialog, doing it from a command line program bypasses the need to distribute your game's source because you aren't linking to it in the form of a static or dynamic library. I'm not completely against rethinking whether this extension should be in enigma, but I do know hugar seemed to get really excited about it, I want Josh's and Robert's opinion before I call it quits.

Interestingly enough, Rust uses mpv media player, although they chose to go the libmpv route: https://docs.rs/mpv/0.2.3/mpv/

fundies  

I don't particularly want that extension either. I was just saying I'd prefer you'd reuse that functionality if you're going to insist on it. Anyways, just displaying a video in the window is no good. A proper video extension should be able to render to a texture in game for stuff like displaying a video on a TV asset in the game but this will require direct usage of a c++ or c library.
time-killer-games  

I think personally this asset is pretty decent for what most real games need videos for; mainly animated logos and brief cut-scenes which pretty much always cover the entire screen, but that's just my opinion. Most games really don't need anything that fancy. It would also be pretty simple to add a custom pause feature but I haven't gotten around to that just yet. I've rarely seen a video rendered to a surface in games I've seen that use videos.

Especially for games that are 2D.

time-killer-games  

@fundies I think you will like this, I have written a video player using DirectShow with some help from someone from the BlitzMax community a while back, I think on Windows what I'm going to do is do a check for the existence of mpv.exe in the working directory and if it does not exist I could use the DirectShow code as a fallback. Trying to compensate for the fact you don't want me doing this. I'm also happy to move the process code into platforms, that way our widgets and other stuff can use it.
fundies  

@fundies I think you will like this, I have written a video player using DirectShow with some help from someone from the BlitzMax community a while back, I think on Windows what I'm going to do is do a check for the existence of mpv.exe in the working directory and if it does not exist I could use the DirectShow code as a fallback. Trying to compensate for the fact you don't want me doing this. I'm also happy to move the process code into platforms, that way our widgets and other stuff can use it.

You misunderstand, I'm all for a video extension but I just don't want you launching exes to do it. You should be using ffmpeg or whatever to render to a texture directly. Robert or I will likely write an extension to do this eventually but if you want to attempt to do it properly, I will answer what questions I can.

time-killer-games  

@fundies alrighty, I added the direct show code, and even threw in some error checking, (just a little).
codecov[bot]  

Codecov Report

Merging #2107 (f951e18) into master (707282b) will decrease coverage by 0.05%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2107      +/-   ##
==========================================
- Coverage   33.03%   32.97%   -0.06%     
==========================================
  Files         197      197              
  Lines       19059    19051       -8     
==========================================
- Hits         6296     6283      -13     
- Misses      12763    12768       +5     
Impacted Files Coverage Δ
...LL/Universal_System/Resources/sprites_internal.cpp 74.50% <0.00%> (-1.86%) ⬇️
...em/SHELL/Graphics_Systems/General/GSprimitives.cpp 67.54% <0.00%> (-1.76%) ⬇️
...Asystem/SHELL/Graphics_Systems/General/GSmodel.cpp 47.41% <0.00%> (-0.47%) ⬇️
...ystem/SHELL/Universal_System/Resources/sprites.cpp 93.18% <0.00%> (-0.16%) ⬇️
.../SHELL/Graphics_Systems/OpenGL-Common/textures.cpp 97.91% <0.00%> (-0.09%) ⬇️
ENIGMAsystem/SHELL/Universal_System/nlpo2.h 100.00% <0.00%> (ø)
...NIGMAsystem/SHELL/Universal_System/image_formats.h 66.66% <0.00%> (ø)
...HELL/Universal_System/Resources/sprites_internal.h 86.66% <0.00%> (ø)
...system/SHELL/Graphics_Systems/General/GSsprite.cpp 10.92% <0.00%> (+1.13%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 707282b...ede2967. Read the comment docs.

time-killer-games  

@RobertBColton @JoshDreamland anything you want me to add to TODO / FIXME list?
RobertBColton  

@time-killer-games we're not too sure about this extension.

The reason is because most video player extensions already do this process API stuff. It's also something that the user is capable of doing in the GML/EDL API already. We don't really want infinite extensions in the default ENIGMA setup. The real meat and potatoes for an official video API extension we would like should hook into our graphics like my original DirectShow extension was planned/going for. In other words things like video_draw, video_get_surface, and etc so you could do things like draw a video on a 3D cube are what is most important for a game engine to provide. I was already planning to get back on this at some point and also things like ffmpeg, but have just been busy with higher priority tasks.

Not completely blocking it yet but letting you know what the current thoughts are.

time-killer-games  

The video player on Windows uses DirectShow so it really isn't possible to do this with enigma already unless you were to use mpv media player like on Linux / FreeBSD but I'm not doing that here; only on Linux / FreeBSD. That and there's the fact you can't use video_stop() with execute_shell() at all nor can you with the current DirectShow extension we have which is also broken by comparison. mpv media player is just a cheap work around for Linux and FreeBSD I wrote until I learn how to use libmpv instead which is basically an easier to use ffmpeg wrapper library. The plan was already for that to be temporary and be replaced quickly.

Whatever happened to that thing fundies said, "code is better than no code". The way I see it, a fully functional & not-broken video player is far better to have now than to wait another 10+ years when ENIGMA might not even exist relevantly anymore.

You guys really put off a lot of progress enigma could potentially make by disliking good ideas and rejecting useful code. I'm not upset or anything, I just think you guys are more harm to yourself and to enigma than anyone else could possibly be for you.

A lot of very basic features that everyone has been desperate for YoYo to add to GameMaker I am solely responsible for adding back via extensions as well as adding those things to enigma. File dialogs, non-broken video playback, access to the shell and external programs, even something as basic as a simple window icon on Linux - probably never would've been added back to GameMaker cross-platform or ENIGMA if I didn't step in and see the demand and actually listen to what people want.

You guys have done a lot more than I have, I'm fully aware of that, but I tend to add things to enigma and gamemaker based on what everyone on the GMC has been asking for repeatedly and with a lot of demand I see this stuff and know what's wanted and what should be higher on our priority list. Things like adding the protobuf for example was a huge waste of time. It caused more problems over the years than anything beneficial. Most people on the GMC don't even know what protobuf is. I don't, honestly...

Lonewolff has been making a lot of sales on his expensive video player extension which has a lot of the features you guys wanted. Something you must've missed, it is technically possible to do what you are after with using surfaces and whatnot using the exact method I am utilizing to play videos on Linux and FreeBSD, see this for reference, it can grab an invisible window to the specified pixmap as a texture. https://stackoverflow.com/questions/19912851/get-pixmap-out-of-xlib Not saying this approach is best w/e.

time-killer-games  

@RobertBColton want to hear another fact about shark poop?
RobertBColton  

We'll see, I suppose that's reasonable under the premise that at the time we have a full proper video extension in ENIGMA, that can draw to 3D cubes and such, you don't complain about us deleting this extension.
JoshDreamland  

It might just be time to add better infrastructure for organizing extensions. We're starting to hit a real problem with the number we have—it's more than a typical person is going to read through and process.

It's true that having some sort of video player code is better than having none, for someone who wants to play video files. But playing video files is a bad idea. Even if we offer perfect support for embedding video files in the game window, we're still doing users a disservice by encouraging them to hack together some kind of recording to play for cutscenes instead of utilizing tools in their game engine to produce one properly in-game.

With users embedding video files instead of doing proper animations, we (and their users) will be storing and transferring hundreds of megabytes of video files for a game that really just needed to move a camera around a few models or send a few sprites across the screen in a specific sequence. This is what timelines are for. Admittedly, we could use better animation sequencing resources. Particularly the kind that let you see what you're doing while you're doing it. Even Yoyo has noticed that one...

The point is, as a game engine, we should be facilitating good approaches to practical problems. The pushback you see from a lot of us is the result of asking about problems with a particular solution to the actual problem, rather than tackling the actual problem in a careful manner.

In this case, the real problem is "how do I make cool cutscenes in my game." But the problem users are coming to you with (or the question you keep searching out users asking) is "AAARGH HOW I EMBED AVI FILE????//" And then the problems you bring to the table are innumerable (but here's a start):

  1. DirectShow is Windows-only and only supports Microsoft codecs
  2. There's not an end-all-be-all video player on non-Windows platforms
  3. There are window focus and presentation issues when launching an external video player
  4. The video player you invoke might already be open with a playlist (paused or playing), which will mean your request is scheduled onto that player instead of actually being carried out
  5. The command you fire will return immediately for some players because the launcher spawns a child process

We've now turned one serious problem into five problems not worth solving. This is a waste of energy. Robert has pitched a more tenable (yet still not ideal) approach to playing video files, but the actual problem we should be talking about is "how do I make a great cutscene?"

There are lots of answers in this space! Blender has great animation support. If we go the video approach, we encourage users to use Blender to turn 65KB of Blender files into 300MB of cutscene video. Blender is open-source software... its render library is probably smaller than the video, assuming it's not smaller than ffmpeg.

And there are answers for 2D sequencing, too, but I'm not aware of a great one that's free. Maybe this is a place ENIGMA could shine. I've seen some proofs of concept on YouTube, but none of them have reached a viable product, as far as I know.

So, in a nutshell, more functionality isn't a bad thing, but we want to at least try to guide users toward what the rest of the industry does, rather than encourage them to do the first hack that comes to mind, as terrible as it is for their infrastructure and users.

time-killer-games  

@JoshDreamland the system2 method is supposed to return immediately. The while loop after that call is what makes it blocking.

However the video_play function runs process_execute in a different thread anyway to keep it async.

JoshDreamland  

You haven't arrived at that problem yet. It will make sense when you arrive at it in a few weeks, after you've "fixed" the focus issue.

The point isn't that there are problems. Problems can be solved. The point is that you've picked the solution with the most problems because it resonated with you. We should pick a more tenable solution.

time-killer-games  

@JoshDreamland what Linux distro has that problem? I'd like to install it so i can debug a fix for that. Not really for this extension but for my process information one for GameMaker which also uses that code. This would be useful for me to fix regardless of whether this extension ever makes it into enigma.

Also, how exactly do you expect me to use this in my game? Should i convert it to PNG and making it a million times bigger in filesize and make it have audio that plays out of sync? https://www.youtube.com/watch?v=74OBPJUNnqY

The focus issue is simple, make the winow not embedded, make it invisible on creation, and then make it into a texture like I mentioned: https://stackoverflow.com/questions/19912851/get-pixmap-out-of-xlib

That doesn't change the fact I don't really have any use for skipping videos. The ones I play are short and just a splash logo played only once at game startup, and a lot of people do that.

RobertBColton  

@time-killer-games The answer to your last question is the same as the answer to the question "How did you render that video in the first place?"

He's saying we can have animation sequences, and blender like CGI and stuff. Most games are actually doing that now, in-game rendered cutscenes.

time-killer-games  

I misunderstood lol. Yeah i rendered the video with Bryce and enigma doesn't have 3D good enough to look like that.

I guess we'll add sequences in 20 years. Better 3D in 30 years. Doesn't change what we don't have now but could have at least something now if we cared enough to allow it.

time-killer-games  

@JoshDreamland I've tested this POSIX code which is also used by my process info extension on MacOS, FreeBSD, and a lot of different Linux distro's. I have yet to see what you were trying to say. I would like to fix it as soon as possible but I have no idea what to do if I can't reproduce it. Sorry for asking so much, I'm just really anxious to get it fixed. drops mic
time-killer-games  

4. The video player you invoke might already be open with a playlist (paused or playing), which will mean your request is scheduled onto that player instead of actually being carried out

mpv media player doesn't work that way. It was designed specifically for use with the command line in mind.

Screenshot_20200818_232611

JoshDreamland  

There isn't a sequence of fixes that will put the extension in a perfect state. If you try to fix the focus issue by waiting for the video player process to terminate, you'll find that it frequently terminates immediately upon scheduling playback with a separate player process. So your focus fix would actually just hide the video in that case.

In the case of your youtube video, I see a single camera pan motion with a model that its either static or stretching vertically. The camera is zoomed out far enough that you could probably just pull that off with purely 2D sprites, but the obvious solution is to enter 3D mode, draw the lava floor and your stretching model, start the camera object moving downward, and set an alarm for 60 frames to head back to the title screen.

Launch an instance of mpv playing a song and then run your game. If it doesn't decentralize, try VLC.

time-killer-games  

@JoshDreamland alternatively people could just use keyboard_check_direct() to skip the video on linux. Although it would work regardless of the window that's active. In which case window_has_focus could be used if it returns true, not sure if it would. Depends on how robert wrote that feature.
RobertBColton  

The keyboard check direct on Linux is fake, it's not really direct and so that won't work. I did some research before and i'm not even really sure if that function is possible on Linux. Regardless it's never useful for anything, even on Windows where it's implemented correctly. It can miss key presses and releases because it's not message based and works below the hardware interrupt level.

Also ENIGMA needs a command line argument parser for options like the one we use in emake. We should provide default options to all games, one of which would be to skip cutscenes which a lot of games provide a command line option for. The user should be allowed to remove those default options though, so maybe we could just include them in configs and allow deleting the default options there.

time-killer-games  

@RobertBColton we already have command line parsing. We have parameter_string and parameter_count.
RobertBColton  

That's basic command line parsing, those aren't options like boost options.
time-killer-games  

@RobertBColton

MacOS has a Darwin API that did exactly what you are referring to, and since Darwin is open source, and likely uses posix api underneath for that API, we could very easily borrow code from that if it isn't substantial for non-mac platforms. I've used that api in the past but I'm having trouble finding it atm. Better yet, we could do the same thing with boost, we just need to credit the authors with the appropriate license header in tact. I don't think we should ship the entire boost dependency though, we should only get the relevant code we need since a lot of boost's stuff we are replacing with other things.

time-killer-games  

@JoshDreamland I say we merge it with a disclaimer added to the About.ey to contact samuelvenable@hotmail.com for support and not the enigma development team, because I am the only one who cares to maintain it and provide support such as answer questions like "y it not AVI?" and your other concerns along those lines aren't a relevant burden to any of you if people are coming to only me with the questions and help/feature requests. What say you?

Pretty please with a COVID-19 infected cherry on top?

Then we can delete this shit and replace it with a better video player once one has been written. But please attempt to conform the new api as closely to the original as you can, to avoid breaking games. Add functions for doing cool extra things like rendering to a surface. But the only function that already exists that would need EDL API changing for the better video player is the second argument of video_play could simply be removed and the game window would be assumed rather than specified manually in that argument. I think that's a pretty logical way to go about it if you are of any mind.

fundies  

Why do you need it to be in the repo. You can just tell users to git clone tkg.github/extension ENIGMASystem/Universal/Extensions/videocrap
time-killer-games  

@fundies also why you guy like and merge Robert's broken and windows-only directshow video player but not mine that isn't broken and working on 3 platforms and therefore is much less limiting?

At the very least we can take out the perfectly working Linux and FreeBSD code for no good reason and replace Robert's broken direct show with my directshow code that doesn't crash and his also has zero error reporting.

All of the complaints you have with my windows direct show player are evenly applied to his if not more of an issue with his due to the fact it crashes and doesn't throw proper errors.

Seems backwards to me. In every way.

I find favoritism is a lot like a disease. It prevents you to think clearly and rationally.

I try to help you guys but you're so full blown arrogant you don't appreciate anything I selflessly do on my spare time. There are literally several dozen people who are all trying to tell me I shouldn't have to deal with this bs and leave you guys to your own demise. I get appreciated everywhere on the internet except mostly by this community of 3 people. I can't make everyone happy, it's a shame the 3 people I can't make happy are the 3 in charge of such an otherwise fantastic project.

Please sign in to post comments, or you can view this issue on GitHub.