Is asm volatile ("csrr %0, mhartid" : "=r"(hartid)); only allowed inside Verilator/VCS simulation?

[JL102]

Hi,

These two lines work effectively to identify the current processor ID when running a Verilator simulation:

  unsigned long hartid = 0;
  asm volatile ("csrr %0, mhartid" : "=r"(hartid));

but when I am running Fedora on firesim (still on a Rocket core, though), and then I attempt to run the same lines, I get an error Illegal instruction (core dumped). Is that instruction only allowed when doing a bare metal simulation in Verilator/VCS?

Here's the hello world:

#include <stdio.h>

int main(void) {
  unsigned long hartid = 0;
  asm volatile ("csrr %0, mhartid" : "=r"(hartid));
  printf("Hello world from core %d\n", hartid);
  return 0;
}

And here's the output:

[root@fedora-riscv ~]# ./hello
[ 1098.148128] hello[340]: unhandled signal 4 code 0x1 at 0x00000000000110e0 in hello[11000+1000]
[ 1098.148186] CPU: 0 PID: 340 Comm: hello Tainted: G           O       6.2.0-297961-ge3e5d275a34f #10
[ 1098.148210] Hardware name: freechips,rocketchip-unknown (DT)
[ 1098.148225] epc : 00000000000110e0 ra : 0000003fad554364 sp : 0000003fd28b4980
[ 1098.148437]  gp : 0000000000014800 tp : 0000003fad6783a0 t0 : 0000003fad6a32b0
[ 1098.148663]  t1 : 0000003fad55411c t2 : 0000000000000003 s0 : 0000003fd28b49a0
[ 1098.148889]  s1 : 0000000000000001 a0 : 0000000000000001 a1 : 0000003fd28b4b28
[ 1098.149114]  a2 : 0000003fd28b4b38 a3 : 0000000000000000 a4 : 0000003fd28b49c0
[ 1098.149340]  a5 : 00000000000110d4 a6 : 0000003fad670198 a7 : 1263636a677c2f40
[ 1098.149566]  s2 : 0000000000000000 s3 : 0000000000013e28 s4 : 00000000000110d4
[ 1098.149792]  s5 : 0000003fd28b4b38 s6 : 0000000000013e28 s7 : 0000003fad6a1d28
[ 1098.150017]  s8 : 0000003fad6a2050 s9 : 0000002ac694f7d4 s10: 0000002ac694f7e0
[ 1098.150244]  s11: 0000002ac696b070 t3 : 0000003fad68f452 t4 : 0000000000027398
[ 1098.150469]  t5 : 0000000000021028 t6 : 0000000003d8f538
[ 1098.150635] status: 8000000200006020 badaddr: 00000000f14027f3 cause: 0000000000000002
Illegal instruction (core dumped)

[yuanjin0928]

Hi, did you find the solution. I met a issue similar to yours. I can't read mcycle as well as mhpmcount.

[JL102]

I wasn't able to find out a way to run that assembly command, no. To read the true hart ID of a core, which was my reason for running that assembly command I made a workaround that stored the hart ID as a "hardcoded" register in my RoCC module. (by "hardcoded" I mean that during the stage when the Chisel is converted into Firrtl [not sure what the correct term is - Elaboration maybe? Compilation?] I assign a Scala variable to the module, which is then turned into a hardware register.

Something along these lines (I copied over what's necessary from my module to get the point across; I don't think this will actually compile & run if you take the code ver batim)

class MyModule(val enableHartIdCommand: Boolean)(implicit p: Parameters) extends LazyRoCC (opcodes) {
  // This is overridden in CanHaveMyModule. 99.U is a placeholder because then it'd be obvious if it's not set properly.
  // The value is for the doGetHartid instruction, in the case where Firesim's CPU ids differ from the hart ids.
  var hartId = 99.U;
  class Impl extends LazyRoCCModuleImp(this) {

    // (Skipping a bunch of other code, e.g. other doX/doY instruction variants)
    val doGetHartId = io.cmd.bits.inst.funct(4) === true.B

    when (doGetHartId) {
      // This bit is simplified; in my actual module I have a state machine and
      // instead of directly setting io.resp.valid, it goes to the "respond" state
      //  The important part is the := hartId; when turned to firrtl it becomes "hardcoded"
      io.resp.bits.data := hartId
      io.resp.valid := true.B
    }
  }
}

// this trait is added in DigitalTop.scala
trait CanHaveMyModule {
  this: HasTiles =>
  val mqs = tiles.flatMap { t =>
    t match {
      case r: RocketTile =>
        r.roccs.collect { case mq: MyModule =>
          if (mq.enableHartIdCommand) mq.hartId = t.hartId.U
          t.hartId -> mq
        }
      case b: BoomTile   =>
        b.roccs.collect { case mq: MyModule =>
          if (mq.enableHartIdCommand) mq.hartId = t.hartId.U
          t.hartId -> mq
        }
    }
  }.toMap
}

class WithMyModule(enableHartIdCommand: Boolean = false) extends Config((site, here, up) => { case BuildRoCC =>
      up(BuildRoCC) ++
        Seq((p: Parameters) => {
          val myModule = LazyModule(new MyModule(enableHartIdCommand)(p))
          myModule
        })
    })

(The reason I set enableHartIdCommand to a parameter is because if it's enabled, then Firrtl can't dedupe the verilog for each instance of my module, since the code is different. So I only enable it when i'm building for firesim.)

Unfortunately, I don't know how to read mcycle or mhpmcount, sorry.

[yuanjin0928]

ah, I see. I was thinking cssr instruction is a privileged instruction which might restrict us from using it in a user space. I don't know. I do see some guy use this instruction in a c++ program, like this one https://github.com/ccelio/riscv-hpmcounters.
Maybe I can try autocounter to get the information of mcycle. Thank you for sharing your work. Even it can not used in my case, it still provides inspirations if I will meet the same issue in the future.