Measuring performance in a hardware-agnostic way

3 years ago by Sekoia to c/advent_of_code

Is there a way to measure performance without depending on the hardware, i.e. two entirely different computers get the same score for the same code?

I could probably run the program on a server or something, but something local feels more reliable.

FuzzChef 7 points 3 years ago

Yes, you can consider the algorithmic complexity of your code, which is independent of any hardware.

path: 0 5744144, hotness: undefined, score: 7, children: 2
Sekoia 2 points 3 years ago

Sure, but that makes a lot of optimizations nonexistent (e.g. cloning the input on an O(n) algorithm is "free")

path: 0 5744144 5744695, hotness: undefined, score: 2, children: 1
damium 4 points 3 years ago

You don't need to use big-O. You can calculate the full complexity in algebraic notation. It's just a lot more work as you don't get to discard terms.

path: 0 5744144 5744695 5750748, hotness: undefined, score: 4, children: 0
stifle867 3 points 3 years ago

Im theory you could try counting CPU cycles or something analogous, or algorithmic complexity as the other commenter mentioned. You could also try to measure it in comparison to different code.

Outside of that, and in any practical sense, no you can't. Performance measurements are dependent on the machine as much as the software.

If a car is hardware, and the driver is software, how would you measure the performance of the driver in the different cars? The only way I can think of is if you had 2 different drivers and could compare their times in both cars. If driver 1 is 2x as fast in car 1, and 2x as fast in car 2, you could say driver 1 has a score 2x higher than driver 2.

path: 0 5746949, hotness: undefined, score: 3, children: 5
Sekoia 1 point 3 years ago

One option could be to emulate a full CPU and run it on that, and count the cycles on that. One issue is linking/syscalls. Maybe compiling to smth like WASM could work?

path: 0 5746949 5747295, hotness: undefined, score: 1, children: 3
stifle867 1 point 3 years ago

I would be interested to hear if you get far down that road. It sounds infinitely more difficult than running code and timing it.

Are you able to expand what problem lead you to this question (out of curiosity)?

path: 0 5746949 5747295 5747380, hotness: undefined, score: 1, children: 2
Sekoia 1 point 3 years ago

Friend and I are comparing times, but since he uses Jai, I can't compile his.

path: 0 5746949 5747295 5747380 5748777, hotness: undefined, score: 1, children: 1
stifle867 1 point 3 years ago

As an alternative could he compile your program?

path: 0 5746949 5747295 5747380 5748777 5760271, hotness: undefined, score: 1, children: 0
FuzzChef 1 point 3 years ago

CPU cycles depend on the instruction set of the hardware. This would also require compilation without any optimization to avoid even software dependency.

path: 0 5746949 5756220, hotness: undefined, score: 1, children: 0
vole 2 points 3 years ago

perf and valgrind might be good places to start. Although, some programs aren't going to have the exact same executed instruction count between runs, and it's possible that executed instruction count can depend on the exact CPU that's running. You can probably mitigate the latter by running valgrind and the program inside of QEMU.

path: 0 5861656, hotness: undefined, score: 2, children: 1
Sekoia 1 point 3 years ago

Thank you! That's exactly the kinda thing I was looking for!

path: 0 5861656 5864359, hotness: undefined, score: 1, children: 0
dns 1 point 3 years ago

I believe everyone gets different input and needs to produce a different result. There can be multiple solutions. Do you process as you are going, or do you parse and build a data structure to process later so you loop through the results multiple times.

path: 0 5744289, hotness: undefined, score: 1, children: 0
advent_of_code
advent_of_code

@programming.dev

login for more options
1248
184
1

An unofficial home for the advent of code community on programming.dev! Other challenges are also welcome!

Advent of Code is an annual Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.

Everybody Codes is another collection of programming puzzles with seasonal events.

EC 2025

AoC 2025

Solution Threads

M T W T F S S
1 2 3 4 5 6 7
8 9 10 11 12

Visualisations Megathread

Rules/Guidelines

  • Follow the programming.dev instance rules
  • Keep all content related to advent of code in some way
  • If what youre posting relates to a day, put in brackets the year and then day number in front of the post title (e.g. [2024 Day 10])
  • When an event is running, keep solutions in the solution megathread to avoid the community getting spammed with posts

Relevant Communities

Relevant Links

Credits

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

console.log('Hello World')

go to feed...