Exposure Time is not controllable with a Basler Camera

[eddybl]

Hey,

so first of all I think this issue might be related to areaDetector/ADAravis#14 and #23

We recently added a new Basler Camera (acA1920-25gm), the first of its type for us. So we got the XML file to set up the .template file etc.
It also mostly works, but at some point we realized that we cannot set the Exposure Time. We can set it and also the readback value changes, but it has no effect on the camera.

It comes basically down to these two records which are part of the auto-generated .template file:

record(ao, "$(P)$(R)GC_ExposureTimeAbs") {
  field(DTYP, "asynFloat64")
  field(OUT,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))GC_D_ExposureTimeAbs")
  field(PREC, "3")
  field(DISA, "0")
}
record(ao, "$(P)$(R)GC_ExposureTimeRaw") {
  field(DTYP, "asynInt64")
  field(OUT,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))GC_I_ExposureTimeRaw")
  field(DISA, "0")
}

With arv-tool-0.8 -n "Basler-acA1920-25gm-***" features | grep -i ExposureTime I get:

Float       : 'ExposureTimeAbs'
Integer     : 'ExposureTimeRaw'
    EnumEntry   : 'ExposureTime'
Float       : 'ChunkExposureTime' (Not available)
Float       : 'AutoExposureTimeAbsLowerLimit'
Float       : 'AutoExposureTimeAbsUpperLimit'

We did not have any issues with previous basler cameras, but it seems like this Camera actually runs on Firmware 4.X, whereas our older cameras run on Firmware 3.7.
I also created an issue with Basler support about this. Part of the relevant response was:

One issue might be the parameter name, “ExposureTimeRaw” is considered deprecated and should be replaced by “ExposureTime”:
https://docs.baslerweb.com/pylonapi/cpp/pylon_advanced_topics#list-of-changes

Is there anything else I can try?

[MarkRivers]

The documentation in the Basler list of changes says there should be a Float feature called ExposureTime. However, arv-tool shows that there is not. You should ask them why.

This is a search for the string Name="ExposureTime in a Point Grey xml file.

corvette:areaDetector/ADGenICam/xml>xmllint --format temp.xml | grep 'Name="ExposureTime'
  <Float Name="ExposureTime" NameSpace="Standard">
  <Float Name="ExposureTimeAbs" NameSpace="Custom">

Note that it has a feature called ExposureTime, and also one called ExposureTimeAbs.

The existing Basler files have these entries:

corvette:areaDetector/ADGenICam/xml>grep '<Float Name="ExposureTime' Basler*.xml
Basler-acA1300-30gm.xml:  <Float Name="ExposureTimeAbs" NameSpace="Standard">
Basler-acA2000-50gmNIR.xml:  <Float Name="ExposureTimeAbs" NameSpace="Standard">
Basler-acA2040-35gm.xml:  <Float Name="ExposureTimeAbs" NameSpace="Standard">
Basler-acA2440-20gm.xml:  <Float Name="ExposureTimeAbs" NameSpace="Standard">
Basler-scA1300-32gm.xml:  <Float Name="ExposureTimeAbs" NameSpace="Standard">
Basler-scA1300-32gm.xml:  <Float Name="ExposureTimeBaseAbs" NameSpace="Standard">
Basler_piA640_210gm.xml:  <Float Name="ExposureTimeAbs" NameSpace="Standard">
Basler_piA640_210gm.xml:  <Float Name="ExposureTimeBaseAbs" NameSpace="Standard">

Note that none of them have a Float feature called ExposureTime, but they all have a feature called ExposureTimeAbs. Your new camera is the same.

This should be fine because ADGenICam maps either ExposureTime or ExposureTimeAbs to the ADAcquireTime parameter:

    // Make a single parameter that maps to either ExposureTime or ExposureTimeAbs
    features = {{"ExposureTime",    GCFeatureTypeDouble},
                {"ExposureTimeAbs", GCFeatureTypeDouble}};
    createMultiFeature(ADAcquireTimeString, asynParamFloat64, ADAcquireTime, features);

It never does anything with ExposureTimeRaw.

So I don't understand why this is not working for your camera, since you do have ExposureTimeAbs, and contrary to their documentation, you do not have ExposureTime.

[MarkRivers]

@eddybl I think I may understand why setting the AcquireTime has no effect on the camera. Many cameras have an ExposureTimeAuto feature which automatically sets the exposure time, and overrides any value you set. Are you sure that you have disabled ExposureTimeAuto?

Another possibility is that FrameRate is enabled and its setting will not allow the exposure time you have requested.
For example, AcquireTime=0.1 and FrameRate=20 are not compatible, and frame rate may take priority.

There may be some other feature that also automatically controls the exposure time. Look through the camera-specific OPI screens carefully.

[MarkRivers]

Any progress on this?

[AbdallaDalleh]

@eddybl I think I may understand why setting the AcquireTime has no effect on the camera. Many cameras have an ExposureTimeAuto feature which automatically sets the exposure time, and overrides any value you set. Are you sure that you have disabled ExposureTimeAuto?

Another possibility is that FrameRate is enabled and its setting will not allow the exposure time you have requested. For example, AcquireTime=0.1 and FrameRate=20 are not compatible, and frame rate may take priority.

There may be some other feature that also automatically controls the exposure time. Look through the camera-specific OPI screens carefully.

I would like to confirm this, we faced it with the acA1300-30gm model. We eventually disabled auto exposure and disabled frame rate and controlled the camera through exposure time only (as the application required) with frame rate calculated as expected from exposure time. Could there be something on the network level? Like jumbo frames? it used to be an issue on some Basler models like the scout...

[MarkRivers]

Could there be something on the network level? Like jumbo frames? it used to be an issue on some Basler models like the scout...

I think a network issue like jumbo frames would only affect streaming data, not whether the camera accepts the exposure time value.

[eddybl]

Sorry for the delay, I was sick and afterwards on Holidays

I would like to confirm this, we faced it with the acA1300-30gm model. We eventually disabled auto exposure and disabled frame rate and controlled the camera through exposure time only (as the application required) with frame rate calculated as expected from exposure time. Could there be something on the network level? Like jumbo frames? it used to be an issue on some Basler models like the scout...

Thanks for the tip regarding the frame rate disabling, this is one option we might not have tried yet. I will give it a try on the next opportunity. We want to acquire one image per second based on an external hardware trigger, so only using the exposure time should be enough.

Edit: Still confusing though that setting the exposure time via the Basler software works as expected even with frame rate on etc.

[AbdallaDalleh]

If you want priority for exposure time you should set its auto to Off, that way it will calculate frame rate from exposure time. If it is set to continuous, if I understand correctly it will calculate exposure time from frame rate assuming frame rate is enabled. This is what is working for us:
Control using exposure time: Exposure auto = OFF | Frame rate enable = NO
Control using frame rate: Exposure auto = Continuous | Frame rate enable = Yes

[MarkRivers]

If it is using a hardware trigger then the frame rate should be ignored.

[MarkRivers]

@eddybl if you are still having problems please post a screen shot of the main ADAravis.adl screen, and each of the camera-specific screens.

[eddybl]

Ok, so the issue still persists: Chaning the exposure time via EPICS does not have an effect on the camera.

Here are some screenshots https://bwsyncandshare.kit.edu/s/sSry8fnm3KxxFiN

• GeniCam.png is the panel we typically use
• Basler1-4.png are the basler specific panels (although not for this camera model)
• PylonViewer.png is the relevant part from the Basler software

What I did:

1. Switched on continoues image taking, but with an external 1 Hz hardware trigger
2. Switched on Auto Exposure, this changed the expusre time to 0.035 s (Exposure-35.png is the picture with 0.035 s)
3. Switched off Auto Exposure
4. Tried to lower Exposure to 0.001
5. The acquired images where still with 0.035 s expsoure

If I restart the IOC, it also reads back the "correct" exposure time (0.001 s), and also using the Basler Software (PylonViewer.png) shows the exposure time which I set via EPICS, but the picture is still acquired with 0.035 s. I will need to change the exposure time in the basler software to have an effect.

Edit: I was wondering if I could possibly try to have a minimal working example using a Python script to directly set the exposure time to the baser camera without EPICS, could this be done? This would finally make clear if it is an aravis/EPICS problem or a basler problem

[MarkRivers]

Why did you not show the camera specific screens for the model you are using? The camera model reported in ADGenICam.adl is acA192025gm. However, the screens you are using are for the acA1300-30gm. That may be why many of the readbacks are undefined? You should really use the screens for the correct camera model.

Also, what top-level screen are you using? If you are using ADAravis then you should use ADAravis.adl as the top-level screen.

In Features Screen #1 it shows the readbacks for ExposureTimeAbs=.001 and ExposureTimeRaw=1000. Those are correct for 0.001 exposure time. Do those values update correctly if you change AcquireTime to 0.005 on ADGenICam.adl. If those readbacks are correct for your camera model then that means that EPICS is correctly setting them.

[eddybl]

Sorry, that is my mistake. I created the records a long time ago and forgot that there is also the option to create the medm panels via script.

Since I am interested in the OPI screens, is this still the way to do that? https://epics.anl.gov/tech-talk/2017/msg01913.php

[MarkRivers]

Since I am interested in the OPI screens, is this still the way to do that? https://epics.anl.gov/tech-talk/2017/msg01913.php

To do the conversions to other OPIs you can just run "make" in GenICamApp/op.

For that to work you need RULES_OPI from here:
https://github.com/EPICS-synApps/support/blob/master/configure/RULES_OPI

You also need to uncomment some lines in this file:
https://github.com/EPICS-synApps/support/blob/master/configure/CONFIG_SITE.linux-x86_64

I have done the conversions to edl, ui, opi, and bob files for the 3 Basler cameras you added.

[eddybl]

Thanks!

So even with the proper panels there seems to be roughly the same amount of channels where the readback is UDF invalid and also directly processing these channels does have no effect.

But I agree in general, it does look like EPICS is doing its job correctly. If I find some time I will try to play around with one of our spare basler cameras and maybe I can figure out if it is maybe firmware dependent...

[MarkRivers]

In Features Screen #1 it shows the readbacks for ExposureTimeAbs=.001 and ExposureTimeRaw=1000. Those are correct for 0.001 exposure time. Do those values update correctly if you change AcquireTime to 0.005 on ADGenICam.adl. If those readbacks are correct for your camera model then that means that EPICS is correctly setting them.

What do you observe for these when you change AcquireTime?

Please post a screen shot of ADAravis screen.

[eddybl]

Yes, the ExposureTimeAbs and ExposureTimeRaw readbacks change correctly if I adjust the AcquireTime

here is a link to the ADGenICam and ADAravis panels next to each other (I changed the value to 0.06): https://bwsyncandshare.kit.edu/s/a2RdXGWHoo37eXt

[LeeYangLBLBCS]

I'm having the same problem with exposureTime. I have a Basler model acA1920-25gc.
screen shot attached:

[MarkRivers]

@LeeYangLBLBCS did you generate .template files and OPI screens for this specific model?

Please post screen shots of all of the camera specific features screens.

[MarkRivers]

@LeeYangLBLBCS when you collect images under the conditions shown above what is shown for the Image rate on the screen?

Please also attach a ,txt file with the output of the following command at the IOC shell.

asynReport 2 SP1

That will show the values of all GenICam features.

[MarkRivers]

@LeeYangLBLBCS please change the Poll features scan time to 1 second in the Status part of the screen. Then post a screen shot of ADAravis.opi when acquisition is on.

You are also converting Mono8 frames to Mono16Low. Is that intentional? It will slow things down with no gain in information.

[LeeYangLBLBCS]

asynReport output is very long, so I have put it in a text file and attached here, together with the output of makeDb.py and makeAdl.py related output.

Basler_setup.txt
I will post Adaravis.adl and Basler feature1-5.adl specific screens in next post.

[LeeYangLBLBCS]

The pixel format conversion only has two possible choices:
Mono16Low
Mono16High.

[MarkRivers]

@LeeYangLBLBCS this time your screen shot of ADAravis does not show the problem with AcquireTime, the setpoint and readback are both 1 second. What did you do to get them to agree? The asynReport output contains these 3 features for ExposureTime.

      Node name: ExposureTimeAbs
          value: 1000000.000000
      asynIndex: 77
       asynName: ACQ_TIME
       asynType: 4
  isImplemented: true
    isAvailable: true
     isReadable: true
     isWritable: true
        minimum: 35.000000
        maximum: 9999990.000000

      Node name: ExposureTimeAbs
          value: 1000000.000000
      asynIndex: 401
       asynName: GC_D_ExposureTimeAbs
       asynType: 4
  isImplemented: true
    isAvailable: true
     isReadable: true
     isWritable: true
        minimum: 35.000000
        maximum: 9999990.000000

      Node name: ExposureTimeRaw
          value: 1000000
      asynIndex: 248
       asynName: GC_I_ExposureTimeRaw
       asynType: 2
  isImplemented: true
    isAvailable: true
     isReadable: true
     isWritable: true
        minimum: 35
        maximum: 9999990
      increment: 1

They all look fine, they are all 1000000 microseconds, which is 1 second. The only thing that seems wrong is the "Image rate" value on the ADAravis screen. It should be 1 (frame/s) but it is 0.

Please set the AcquireTime to 0.1 second, start acquisition, and send a screen shot of ADAravis and the features screen #2 which has the ExposureTime values. Does AcquireTime_RBV agree with AcquireTime? Also run camonitor on the ArrayCounter_RBV so we can see the time between frames.

[MarkRivers]

@LeeYangLBLBCS wrote:

The pixel format conversion only has two possible choices:
Mono16Low
Mono16High.

Sorry, that was my mistake. ADAravis only uses that PV if the pixel format is one of the 12-bit types, so it is ignored when it is Mono8.

[MarkRivers]

It looks to me like you set the AcquireTime to 0.1 second but it is actually exactly 10 seconds. This is a difference of a factor of 100. If you change it to 0.2 or 0.05 what you do observe? Does the time between images change, or it is always 10 seconds?

[MarkRivers]

Please send a screen shot of the features2 screen when AcquireTime is 0.1 seconds.

There is something I don't understand about the values you posted previously. Note that in features screen 2 ExposureTimAbs is 1.0, i.e. it has units of seconds. However, in the asynReport the value is this:

      Node name: ExposureTimeAbs
          value: 1000000.000000
      asynIndex: 401
       asynName: GC_D_ExposureTimeAbs
       asynType: 4
  isImplemented: true
    isAvailable: true
     isReadable: true
     isWritable: true
        minimum: 35.000000
        maximum: 9999990.000000

The value is 1000000, i.e. the units are microseconds. I don't understand this, I cannot find any place in ADGenICam or ADAravis where that value would be changed from microseconds to seconds. Note there is also a maximum value of 9999990.000000, which is 10 seconds. I am wondering if we are setting an exposure time in microseconds, but the camera is interpreting it as seconds, and thus limiting it to 10.0?

[LeeYangLBLBCS]

I changed exposureTime=5, started acquisition, waited for a VERY long time, no image take, gave up.
then changed exposureTime=0.5, started acquistion, camonitor shows same 10 seconds between each frame:

BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 06:53:46.441892 565
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 06:53:56.442429 566
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 06:54:06.442766 567
BL531BSLR1:cam1:AcquireTime_RBV 2022-09-25 06:54:32.903176 5
BL531BSLR1:cam1:AcquireTime_RBV 2022-09-25 06:59:47.686812 0.5
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:00:06.686044 568
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:00:16.686377 569
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:00:26.686834 570
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:00:36.687259 571
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:00:46.687533 572

[MarkRivers]

I don't understand this, I cannot find any place in ADGenICam or ADAravis where that value would be changed from microseconds to seconds.

I was wrong, ADGenICam does convert from seconds to microseconds for the ExposureTime and ExposureTimeAbs features on GenICamFeature.cpp:

double GenICamFeature::convertDoubleUnits(double inputValue, GCConvertDirection_t direction)
{
    double outputValue = inputValue;
    if ((mFeatureName == "ExposureTime") ||
        (mFeatureName == "ExposureTimeAbs") ||
        (mFeatureName == "TriggerDelay")) {
        // EPICS uses seconds, GenICam uses microseconds
        if (direction == GCConvertToEPICS)
            outputValue = inputValue / 1.e6;
        else
            outputValue = inputValue * 1.e6;
    }
    else if (mAsynName == "ACQ_PERIOD") {
        // EPICS uses period in seconds, GenICam uses rate in Hz
        outputValue = 1. / inputValue;
    }
    return outputValue;
}

I wonder if the Basler cameras are not obeying the GenICam specification, and they are using seconds rather than microseconds for ExposureTimeAbs? Please try setting AcquireTime to 1e-6 and see if it acquires for 1 second, and 1e-7 and see if it acquires for 0.1 second.

[LeeYangLBLBCS]

Changing exposureTime to 0.00001 didn't work, the acquistion simply haulted. The ADAravis.adl displays 0, possible because I need to change it's significant digits somewhere to show it.
Then I changed exposureTime to 0.01, then the camera acquires normally again with frameRate ~0.18 seconds:

BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:13:36.716970 649
BL531BSLR1:cam1:AcquireTime_RBV 2022-09-25 07:13:51.694385 1e-06
BL531BSLR1:cam1:AcquireTime_RBV 2022-09-25 07:13:51.749301 3.5e-05
BL531BSLR1:cam1:AcquireTime_RBV 2022-09-25 07:15:44.424320 0.01
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:47.437164 650
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:47.611616 651
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:47.785715 652
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:47.960107 653
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:48.253971 654
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:48.308998 655
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:48.483385 656
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:48.657567 657
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:48.831912 658
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:49.006184 659
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:49.258355 660
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:49.354893 661
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:49.530983 662
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:49.703472 663
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:49.879368 664
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:50.052114 665
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:50.247649 666
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:50.401156 667
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:50.575262 668
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:50.749661 669
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:50.923824 670
BL531BSLR1:cam1:ArrayCounter_RBV 2022-09-25 07:15:51.098253 671

[MarkRivers]

I wonder if the Basler cameras are not obeying the GenICam specification, and they are using seconds rather than microseconds for ExposureTimeAbs?

That actually seems unlikely because the asynReport shows that the minimum and maximum values are 35 and 9999990.000000, which are clearly in units of microseconds. And when you set AcquireTime to 0.5 ExposureTimeRaw_RBV was 500000, which is correct, since Raw is definitely in units of microseconds.

The asynReport output shows clearly that ExposureTime and ExposureTimeAbs are set correctly when you change them in EPICS. FrameRateEnable is no, and ExposureAuto is false. It seems that the camera is behaving in a way that we don't understand.

I see that you tried 0.01 and got 0.18 seconds. The camera may not be capable of 100 frames/s. What is the specified maximum frame rate of the camera? Please try some other values between 0.01 and 1.0 and make a table of the requested values and measured values.

[LeeYangLBLBCS]

The effect of exposureTime on the actual frame rate has a hysterisis.
The following table is the exposure Times and their corresponding measure frame interval from camonitor log.
It looks to me that the camera needs a very small exposure time (0.0001s) to bring back a relatively normal frame rate.
It will stay at a relatively long interval (~0.7 sec) once it gets into that mode.

exposureTime(s)	ArrayCounter_RBV(s)
0.01	0.18
0.02	0.17
0.04	0.19
0.08	0.23
0.1	0.17
0.2	0.18
0.4	0.18
0.5	0.18
0.7	0.7
0.6	0.75
0.5	0.7
0.4	0.701
0.2	0.7
0.1	0.7
0.8	0.66
0.08	0.75
0.04	0.7
0.02	0.7
0.01	0.7
0.001	0.7
0.0001	0.17
0.1	0.18

[MarkRivers]

This is very strange. I have no explanation.

I would suggest contacting Basler to see what the issue could be. You can show them that the readbacks for the ExposureTime and ExposureTimeAbs GenICam features from the camera are correct, but that the camera is not obeying those times in terms of its frame rate.

[MarkRivers]

Any progress on this?

[LeeYangLBLBCS]

I'm waiting for Baser's email response. We are using an alternative without problem, AVT Manto507C. We plan to buy another Basler, such as acA2000, that doesn't have this problem.

…

On Wed, Oct 5, 2022 at 6:04 AM Mark Rivers ***@***.***> wrote: Ant progress on this? — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSYGNBSWKY2A76RTQZOB4TWBV4FDANCNFSM54S4PALA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, M/S 15R0217 Berkeley, California 97320 office:(510)486-7320