Is it working for you too?

Yes

json pls <3

edit: (nevermind)

How do I use it? I need to invert the projection matrix, and I don't want to do it in a shader (faster on the CPU end).

Also, how do you send a matrix to a shader in Enigma?

No it seems correct. Compared it to Model Creator and had same color.

Weird. What shader and what normal matrix did you use? I tried both your code which creates the normal matrix in shader, as well as the modified my code which does it on the CPU. In both cases I cannot get the floor to be blue. This is what I get:

I also tested the surface fix, that is why the surface draws correct side up. But the floor and walls are still wrong. The floor is in x/y plane, so the normal should be in z (blue).

No your picture looks correct. When I was speaking about the floor being blue, I was speaking of outputting JUST the vertex normals, not combined with the normal matrix. When you combine it with the normal matrix then:
-triangles that have a normal face in the direction of your camera are blue
-triangles that face upwards are green

Since you're multiplying the normalmatrix by the in_Normal, it should always be consistent no matter what angle your camera is facing; which it is in your picture

If I output JUST the normals the scene looks like:

I do not think my fix for d3d_draw_floor() is 100% correct. It has correct results for floors, but I think ceilings are off.
[edit]
No it seems correct. Compared it to Model Creator and had same color.

I fixed the wall and floor code in GL3Model.cpp:
http://pastebin.com/9McPLkSU

I also flipped the surface drawing in GSsurface.cpp:
http://pastebin.com/aBgq9hH3

I really don't know if d3d_draw_floor() is correct.

Backface culling is enabled, and I am drawing a floor that correctly shows its face. Since the face is pointing directly upward, its normal would be (0, 0, 1).

So, the in_Normal in the shader should then also be (0, 0, 1).
In my shader, I am outputting in_Normal as the outColor. It's black, so the normal must be negative.



[EDIT]
I turned off culling, and changed the floor drawing to:

Code Select Expand

d3d_draw_floor( 0, 0, 0, 512, 512, 0, background_get_texture(bkg_0), 16, 16 );
Now, the floor is technically a ceiling, but since culling is disabled I can see it as a floor. In the shader, it is a solid blue (0, 0, 1). So, d3d_draw_floor is flipped!
Perhaps in your lighting shader, you were negating the viewposition when it was already negated, which is why it appeared to work.





[EDIT 2]
I made a replica of the map using Model Creator. Here is what the normal buffer looks like:


Here is what it looks like using d3d_draw functions:

I think that the normal matrix isn't calculated properly.
...
I believe that the view matrix isn't properly calculated.

If that would be the case, then model_view matrix would be wrong, as well as model_view_projection matrix. Essentially everything would be wrong then. I did have problems with normal matrix calculation previously though. Right now it's used in lighting engine, and it took me some time to make it look correctly. This is the part that calculates the matrices:

Code (edl) Select Expand

void transformation_update(){
        if (enigma::transform_needs_update == true){
            //Recalculate matrices
            enigma::mv_matrix = enigma::view_matrix * enigma::model_matrix;
            enigma::mvp_matrix = enigma::projection_matrix * enigma::mv_matrix;

            //normal_matrix = invert(transpose(mv_submatrix)), where mv_submatrix is modelview top-left 3x3 matrix
            enigma::Matrix4 tmpNorm = enigma::mv_matrix.Transpose().Inverse();
            enigma::normal_matrix = enigma::Matrix3(tmpNorm(0,0),tmpNorm(0,1),tmpNorm(0,2),
                                                    tmpNorm(1,0),tmpNorm(1,1),tmpNorm(1,2),
                                                    tmpNorm(2,0),tmpNorm(2,1),tmpNorm(2,2));
            enigma::transform_needs_update = false;
        }
    }It seems you try inverting and then transposing, while I think it should be the other way around. To be honest, I have seen both being used, so I never figured out which is correct. Like here it shows the way I did it: http://stackoverflow.com/questions/21079623/how-to-calculate-the-normal-matrix

I will test some.

that external link says:
transpose(invert(viewMatrix));
which is
viewMatrix.invert().transpose();


you're doing
viewMatrix.transpose().invert();

Also, you're supposed to extract it to a mat3 before applying the operations. Though, I am not sure if this has an effect.


In Java I would do it:

Code Select Expand

public static Matrix3f createNormalMatrix(Matrix4f matrix, Matrix3f normalMatrix){

    // extract 4x4 matrix to 3x3 matrix
    extract(matrix, normalMatrix);

    // Invert the matrix
    normalMatrix.invert();

    // Transpose the matrix
    normalMatrix.transpose();

return normalMatrix;
}
--

Regardless, look at my edited post about in_Position.

[should]

Okay I changed the normal shader:

Code Select Expand

// Vertex
in vec3 in_Position;
in vec3 in_Normal;

out vec3 v_Normal;

void main() {
    v_Normal = normalize(normalMatrix * in_Normal);
    gl_Position = modelViewProjectionMatrix * vec4( in_Position.xyz, 1.0);
}

Code Select Expand

// Fragment
in vec3 v_Normal;

out vec4 out_FragColor;

vec3 encodeNormal(vec3 normal) {
    normal = normalize( normal );
    vec3 shifted = (normal+vec3(1.0, 1.0, 1.0))/2.0;
    return shifted;
}

void main() {
    out_FragColor = vec4( encodeNormal(v_Normal), 1.0 );
}

I noticed something odd. I think that the normal matrix isn't calculated properly. What should happen, is that every polygon face you stare at directly should be Blue. Faces that are straight up (floors) should be green, Faces facing right (walls on the left) should be red. However, this isn't the case.

I also tried this to calculate the normal matrix:

Code Select Expand

in vec3 in_Position;
in vec3 in_Normal;

out vec3 v_Normal;

void main() {
    mat3 extractedViewMatrix = mat3(viewMatrix);
    mat3 normalMatrix = transpose(inverse(extractedViewMatrix));
    v_Normal = normalize(normalMatrix * in_Normal);
    gl_Position = modelViewProjectionMatrix * vec4( in_Position.xyz, 1.0);
}

I believe that the view matrix isn't properly calculated. Either that, or in_Normal isn't correctly calculated when using d3d_draw_ commands.




[EDIT]
I believe I have confirmed that in_Normal is incorrect when using the d3d_draw commands!

Code Select Expand

// Vertex
in vec3 in_Position;
in vec3 in_Normal;

out vec3 v_Normal;

void main() {
v_Normal = normalize(in_Normal);
    gl_Position = modelViewProjectionMatrix * vec4( in_Position.xyz, 1.0);
}

Code Select Expand

// Fragment
in vec3 v_Normal;

out vec4 out_FragColor;

void main() {
    out_FragColor = vec4( v_Normal, 1.0 );
}

In my example, every face is black except for 1 wall. This should not be the case. Only two of the walls should be black (negative normal), two walls should have color (one green, one red), and the floor should be blue.

ENIGMA does provide you with normal matrix. "uniform mat3 normalMatrix;" in GLSL.

It does? Well I spent the past 10 minutes arguing with Josh for no reason then.

Also, you can see the two surfaces are upside down. I don't really have an idea what to do in this case. I guess we just can keep it like this and allow people to rotate the projection how they choose. In 2D they will be correct side up, because in recent changes I made the default projection in surface_set_target() to flip the projection. We cannot do that when we want surfaces to be used with 3D.

Because OpenGL flips the y axis so the surfaces are flipped when drawn. An easy fix is to modify draw_surface(...). In my own OpenGL Java engine that I wrote, when I draw a surface like I do in this example, I just draw it with a quad model that has flipped y texture coords. I think that is the most practical fix, as it doesn't disrupt any math down the line, it is solely for viewing purposes.

The example is very well written. Has comments for every line, divides everything in chunks so people can learn more easily. I'm also impressed that you managed to do this even though we haven't documented some of the stuff, like the fact that surface_create() has a third optional parameter enabling depth buffer for that FBO.

Thanks ^^
Robert helped me a bit in the IRC last night.

We could also make a way to assign buffers to surfaces like GL, so you could add a depth texture and use that instead, without making a specific shader for that. I'm not sure how to do it efficiently now though.

To be completely honest I don't know if there would be a major performance gain in doing this. Regardless, I couldn't figure out how to do it in my own Java engine :c

I've known about Enigma for a few years now, this is the first thing I've made in it

It's a simple little shader test containing an 8-bit depth buffer shader, and a normal buffer shader.

Enjoy ^^

Source:
https://www.dropbox.com/s/k4rx5vk40jsrs3v/Shader%20Test.egm?dl=0

Exe:
https://www.dropbox.com/s/4y9amzsubshnk52/shader.exe?dl=0

I don't think that the normal buffer is entirely correct; if Enigma gave me access to the normal matrix then I could guarantee it's correctness

O:

I remember you...
Last time I saw you was ~4 years ago on 64digits O:

http://sandbox.yoyogames.com/games/181293-thesnidrs-geodesic-dome
Why not this?