Test harness should update comments

Reporter: fundies  |  Status: open  |  Last Modified: April 09, 2020, 11:08:24 AM

The test harness should upgrade the regression comments instead of reposting comments everytime.
I am not so sure about how far to go with this one. For the same commit, it would actually be nice yes if EnigmaBot would keep all of the image differences in a single table and just append with new rows for each test. That would allow you to still see some results quickly without having to wait on all draw tests.

Now on the other hand, I don't know if it's a good idea for us to do this across commits. As in, once EnigmaBot finishes commenting for one commit, I don't think it's a good idea to have EnigmaBot edit that comment again on the next commit. That is what Code Coverage bot does, but I don't think it's a good idea for EnigmaBot. The reason has to do with the whole way we design tests. First you design them to fail, and then you make them pass. The only way to show that is to have EnigmaBot post twice, once to say everything failed, and then once to say you fixed everything.

Please sign in to post comments, or you can view this issue on GitHub.