Error checking potentially not computed correctly if CRC and data type fail

[petertorelli]

Reported:

If the seedcrc doesn’t match a known type then total_errors=-1, However if the call to check_data_types()then fails then we could end up with a count of zero and a pass.

	switch (seedcrc) { /* test known output for common seeds */
		case 0x8a02: /* seed1=0, seed2=0, seed3=0x66, size 2000 per algorithm */
			known_id=0;
			ee_printf("6k performance run parameters for coremark.\n");
			break;
		case 0x7b05: /*  seed1=0x3415, seed2=0x3415, seed3=0x66, size 2000 per algorithm */
			known_id=1;
			ee_printf("6k validation run parameters for coremark.\n");
			break;
		case 0x4eaf: /* seed1=0x8, seed2=0x8, seed3=0x8, size 400 per algorithm */
			known_id=2;
			ee_printf("Profile generation run parameters for coremark.\n");
			break;
		case 0xe9f5: /* seed1=0, seed2=0, seed3=0x66, size 666 per algorithm */
			known_id=3;
			ee_printf("2K performance run parameters for coremark.\n");
			break;
		case 0x18f2: /*  seed1=0x3415, seed2=0x3415, seed3=0x66, size 666 per algorithm */
			known_id=4;
			ee_printf("2K validation run parameters for coremark.\n");
			break;
		default:
			total_errors=-1;
			break;
	}
	if (known_id>=0) {
		for (i=0 ; i<default_num_contexts; i++) {
			results[i].err=0;
			if ((results[i].execs & ID_LIST) && 
				(results[i].crclist!=list_known_crc[known_id])) {
				ee_printf("[%u]ERROR! list crc 0x%04x - should be 0x%04x\n",i,results[i].crclist,list_known_crc[known_id]);
				results[i].err++;
			}
			if ((results[i].execs & ID_MATRIX) &&
				(results[i].crcmatrix!=matrix_known_crc[known_id])) {
				ee_printf("[%u]ERROR! matrix crc 0x%04x - should be 0x%04x\n",i,results[i].crcmatrix,matrix_known_crc[known_id]);
				results[i].err++;
			}
			if ((results[i].execs & ID_STATE) &&
				(results[i].crcstate!=state_known_crc[known_id])) {
				ee_printf("[%u]ERROR! state crc 0x%04x - should be 0x%04x\n",i,results[i].crcstate,state_known_crc[known_id]);
				results[i].err++;
			}
			total_errors+=results[i].err;
		}
	}
	total_errors+=check_data_types();

[petertorelli]

This should not be closed. But a future fix is still WIP.

[Sesib]

getting "listcrc" mismatch in Coremark run. What could be the potential issue?

[petertorelli]

You can enable CORE_DEBUG to print useful debugging messages. If you have a platform that passes w/o the crc error, generate debug logs for both systems and compare where things went wrong and debug from there.

[anuani21]

Hi sir,

Am running coremark benchmark in riscv32 bit baremetal. build success. when i load the coremark.bare.riscv, am getting incorrect crc list ,crc matrix and crcstate value for 2k performance run algorithm.

Am getting correct seedcrc value, can you please help me to resolve the issue??

[anuani21]

when i am running CORE_DEBUG also am getting the same issue sir.

[petertorelli]

Setting CORE_DEBUG won't fix the issue, but it will help you understand where the problem might be. If you look at the code, you will see that the values printed in the CORE_DEBUG statements are the same values used to generate the CRCs. If the values differ between a good run and a bad run, you need to investigate why.

[anuani21]

Hi sir,

I have tried the core_debug for examine the problem, but am facing some trap issue.
Can you please help me how to fix the issue sir?
output.log

[anuani21]

Sir,

Can you please share your core_debug log for my reference sir?

[petertorelli]

I notice your matrix size is 11x11. This means different run parameters were specified from the default: ./coremark.exe 0x3415 0x3415 0x66 0 7 1 2000 Try running "out of the box" first to see why your code is failing.

Here is a successful CORE_DEBUG=1 log: run2.log.

[anuani21]

Hi,
Can you please share core debug log for run1.log sir?

[anuani21]

Hi sir, Can you please share the link for 32bit baremetal coremark source in riscv wrapper?

…

On Fri, 23 Sep, 2022, 9:40 pm Peter Torelli, ***@***.***> wrote: I notice your matrix size is 11x11. This means different run parameters were specified from the default: ./coremark.exe 0x3415 0x3415 0x66 0 7 1 2000 Try running "out of the box" first to see why your code is failing. Here is a successful CORE_DEBUG=1 log: run2.log <https://github.com/eembc/coremark/files/9635312/run2.log>. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3G6FF7WSHLJF53YSZQ4GMTV7XI7RANCNFSM4F6EYE6Q> . You are receiving this because you commented.Message ID: ***@***.***>

[anuani21]

Hi sir,

Here I have shared my run2.log. list initalization value is differ from your log.Can you please examine and tell me what is the issue?
validation.log

[gowdanp]

Hi,

I am also facing the same issue as discussed above. I have attached the log file taken on COM port by enabling CORE_DEBUG option. Can you please help me understanding this issue.

Thank you..

coremark.log

[petertorelli]

@anuani21 -

Here is run1.log. Regarding your second question, EEMBC does not have source code for different platforms.

[petertorelli]

@anuani21 & @gowdanp

The fact that idx and data are so far off is hint. This repository is for fixing CoreMark issues, not debugging user problems. It is up to you to root cause the problem and identify if there is a problem with CoreMark that we need to fix. Debugging problems like this can vary. If you look at the valid log files, you can see that the very first out put of the "Initialize list" for run1.log should be:

[0000,8080][0001,7373][0002,7a7a][0003,6161][0004,6868][0005,1717][3402,0505][3406,6161][340e,2121][3412,0505]

But yours is:

[ffff8000,8080][ffffffff,ffff][0502,0505][0512,0505][0c0d,0c0c][0c1d,0c0c][1700,1717][1705,1717][1e03,1e1e][1e13,1e1e][210e,2121][211e,2121][2809,2828][2819,2828][330c,3333][331c,3333][3a0f,3a3a][3a1f,3a3a][451a,4545][4c05,4c4c][5718,5757][5e1b,5e5e][6103,6161][6106,6161][6801,6868][6804,6868][7301,7373][7304,7373][7a02,7a7a][7a07,7a7a]

Ok, why is this? What is your next level of debug results? If you look at the code you can see that list->info->idx and list->info->data16 are initialized to 0x000 and 0x8080 inside core_list_init. After the blocks are added, the list is traversed and the first 20% of the items are assigned i++ and the remaining 80% are assigned a simple pseudo-random number.

Instead of seeing 1, 2, 3, 4... I see 0xffff8000, 0xffffffff, 0x502, 0x512... that right there is a big problem. Obviously assigning i++ isn't going to cause that problem, so it is likely a pointer problem. Is the code using the wrong user defined types for pointers? Is there an assumption about pointer sizes in CoreMark that is invalid. These are just guesses. You need to debug it and file an issue with what is broken, if that is the case.

[anuani21]

Thanks, I'll check them out.

…

On Mon, Sep 26, 2022 at 9:17 PM Peter Torelli ***@***.***> wrote: @anuani21 <https://github.com/anuani21> & @gowdanp <https://github.com/gowdanp> The fact that idx and data are so far off is hint. This repository is for fixing CoreMark issues, not debugging user problems. It is up to you to root cause the problem and identify if there is a problem with CoreMark that we need to fix. Debugging problems like this can vary. If you look at the valid log files, you can see that the very first out put of the "Initialize list" for run1.log should be: [0000,8080][0001,7373][0002,7a7a][0003,6161][0004,6868][0005,1717][3402,0505][3406,6161][340e,2121][3412,0505] But yours is: [ffff8000,8080][ffffffff,ffff][0502,0505][0512,0505][0c0d,0c0c][0c1d,0c0c][1700,1717][1705,1717][1e03,1e1e][1e13,1e1e][210e,2121][211e,2121][2809,2828][2819,2828][330c,3333][331c,3333][3a0f,3a3a][3a1f,3a3a][451a,4545][4c05,4c4c][5718,5757][5e1b,5e5e][6103,6161][6106,6161][6801,6868][6804,6868][7301,7373][7304,7373][7a02,7a7a][7a07,7a7a] Ok, why is this? What is your next level of debug results? If you look at the code you can see that list->info->idx and list->info->data16 are initialized to 0x000 and 0x8080 inside core_list_init. After the blocks are added, the list is traversed and the first 20% of the items are assigned i++ and the remaining 80% are assigned a simple pseudo-random number. Instead of seeing 1, 2, 3, 4... I see 0xffff8000, 0xffffffff, 0x502, 0x512... that right there is a big problem. Obviously assigning i++ isn't going to cause that problem, so it is likely a pointer problem. Is the code using the wrong user defined types for pointers? Is there an assumption about pointer sizes in CoreMark that is invalid. These are just guesses. You need to debug it and file an issue with what is broken, if that is the case. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3G6FFZQJHDHN4FSTQFG23DWAHAPVANCNFSM4F6EYE6Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[anuani21]

Sir,

I have one doubt, In coremark seed values are not getting
correctly(seed1=0, seed2=0, seed3=0x66). Am getting random seed number for
seed1 and seed 2.seed 3 value alone getting correctly as 66 all the
time,but seed 1 and 2 values differing. Is there any idea,what is the issue
sir?
seed.log

[anuani21]

#4 (comment)