I'm not sure it's a good idea to add the gas costs always for all test.

There was an error when running chk_coding_style for commit 1be489917b2431f2d7e93c5319f64529e628325c:

Error: Trailing whitespace found:
test/libsolidity/semanticTests/ecrecover/ecrecover_abiV2.sol:13:// 0x73b1693892219d736caba55bdb67216e485557ea6b6af75f37096c9aa6a5a75f, 

Please check that your changes are working as intended.

We should also add deploy gas costs, or maybe better: code size

I'll change PR now, so that gas is checked only for these tests where expectation is present and will try filtering out tests where we want to run at the beginning.

@cameel Updated
Gas tests will be run only when expectation is present.
I removed gas expectations from all the tests that have less than 100K gas usage (still 600+ tests are affected)
Also, updated to use only latest version of evm for gas usage

I think I addressed all comments, will wait for points we need more feedback.

Maybe it would be interesting to add test cases that are doing gas measurements for all the different evm versions. E.g. setting the same tests that only differ in the defined EVMVersion. With this we could more easily detect if e.g. a new future was accidentally changing the gas-costs of older evm versions. But I'm not sure whether it would really make sense. I just imagine, if someone adds a feature for the latest evm version, that should only influence the gas cost in that particular version, but it also change the gas cost of other version(s), that this could be interesting to see.

@aarlt Regarding what gas costs are added to expectations by enforce feature, I did it in a way that I add only gas costs greater than X(which was set to 100000 gas). So if, for example ir* is missing, it means that it has gas cost of less than 100k gas...

If it is confusing, I'll change it

I just checked out your branch and ran the tests locally. It looks like that I have always different gas measurement results.

If i run test/tools/isoltest -t semanticTests/\* --vm /Users/alex/evmone/lib/libevmone.dylib --no-smt I got this:

semanticTests/constructor_ihneritance_init_order_2.sol: FAIL
  Contract:
    contract A {
        uint x = 42;
        function f() public returns(uint256) {
            return x;
        }
    }
    contract B is A {
        uint public y = f();
    }
    // ====
    // compileToEwasm: also
    // compileViaYul: also

  Gas results missing or wrong, obtained result:
  // constructor() ->
  // gas LegacyOptimized: 138540
  // gas Yul: 232112
  // gas YulOptimized: 153851
  // gas Legacy: 151224

  // y() -> 42
  // gas LegacyOptimized: 22092
  // gas Yul: 22744
  // gas YulOptimized: 22262
  // gas Legacy: 22193

and in the test file

❯ cat ../test/libsolidity/semanticTests/constructor_ihneritance_init_order_2.sol
contract A {
    uint x = 42;
    function f() public returns(uint256) {
        return x;
    }
}
contract B is A {
    uint public y = f();
}
// ====
// compileToEwasm: also
// compileViaYul: also
// ----
// constructor() ->
// gas Legacy: 152088
// gas LegacyOptimized: 138540
// gas Yul: 232112
// gas YulOptimized: 153851
// y() -> 42
// gas Legacy: 22193
// gas LegacyOptimized: 22092
// gas Yul: 22744
// gas YulOptimized: 22262

What could be the reason for this?

@cameel Updated :)