(core/shmif/lua) adjust CLOCKREQ behaviour

This makes a number of changes to how CLOCKREQ behaves across the stack.
First it is no longer forwarded to the Lua layer at all, the handling of
a custom timer callback is all managed internally and only one active is
ever permitted.

Then it adds two additional clock feedback modes, one on a dominant
VBLANK and one on rendertarget presentation feedback. Both of these need
some work that would push into the platform for conveying multiscreen
feedback as well as presentation attached to a non-mapped rendertarget
(e.g. for recording/sharing/offscreen effects).

The presentation feedback is there to measure latency in a normal naive
loop:

source:
1. create frame
2. submit, wait for ACK
3. wait for STEPFRAME (~= frame callback)

sink:
1. gather updates from sources, signal the transfer as STEPFRAME and
   semaphore.
2. compose into sink-output, increment MSC, forward compse as STEPFRAME.
3. signal VBLANK, forward VBLANK as STEPFRAME.

This is adjusted in order to eventually drive X11-PRESENT and Wayland
presentation feedback extensions better than can currently be done. The
full story is still missing actual fences for mailbox and composition
that isn't "sniped" by asynch-submit infinite jobs.

CHANGELOG.md

@@ -8,11 +8,13 @@
  * video\_displaymode expose eotf / coordinates for primaries and contents light levels
  * open\_nonblock can now adopt an existing iostream into a target vid
  * input\_remap\_translation overloaded form for serializing backing keymap
+ * clockreq no longer forwarded for frameserver event handler
 
 ## Core
  * respect border attribute in text rasteriser
  * added frame\_id to external events that pairs with shmif-SIGVID signals
  * optional tracy build for profiling (-DENABLE\_TRACY)
+ * frameserver clock(stepframe) event handling extended (see shmif)
 
 ## Tui
  * nbio asynch type confusion fix (function becomes pcall:userdata)
@@ -51,6 +53,7 @@
  * wire up ext venc resize for compressed video passthrough
  * extend hdr vsub with more metadata
  * dropped unused rhints and rename hdr16f (version bump)
+ * CLOCKREQ extended with options for latching to specific msc/vblank events
 
 ## Decode
  * defer REGISTER until proto argument has been parsed, let text register as TUI

doc/launch_target.lua

@@ -60,7 +60,7 @@
 -- Possible statustbl.kind values: "preroll", "resized", "ident",
 -- "coreopt", "message", "failure", "framestatus", "streaminfo",
 -- "streamstatus", "segment_request", "state_size",
--- "viewport", "alert", "content_state", "registered", "clock", "cursor",
+-- "viewport", "alert", "content_state", "registered", "cursor",
 -- "bchunkstate", "proto_update", "mask_input", "ramp_update"
 --
 -- @tblent: "preroll" {string:segkind, aid:source_audio} is an initial state
@@ -187,11 +187,7 @@
 -- types have been changed. ref:target_input calls that attempts to forward a
 -- masked type will have matching events dropped automatically. The actual
 -- mask details can be queried through ref:input_capabilities.
---
--- @tblent: "clock" (value, monotonic, once) - frameserver wants a periodic or
--- fire-once stepframe event call. monotonic suggests the time-frame relative to
--- the built-in CLOCKRATE (clock_pulse)
---
+-
 -- @tblent: "cursorhint" {message} - lacking a customized cursor using a subseg
 -- request for a cursor window, this is a text suggestion of what local visual
 -- state the mouse cursor should have. The content of message is implementation

src/engine/arcan_conductor.c

@@ -165,6 +165,24 @@ static void step_herd(int mode)
 		TRACE_SYS_DEFAULT, mode, conductor.transfer_cost, "step-herd");
 }
 
+static void forward_vblank()
+{
+	for (size_t i = 0; i < frameservers.count; i++){
+		struct arcan_frameserver* fsrv = frameservers.ref[i];
+		if (fsrv && fsrv->clock.vblank){
+			struct arcan_vobject* vobj = arcan_video_getobject(fsrv->vid);
+
+			platform_fsrv_pushevent(fsrv, &(struct arcan_event){
+					.category = EVENT_TARGET,
+					.tgt.kind = TARGET_COMMAND_STEPFRAME,
+					.tgt.ioevs[0].iv = 0,
+					.tgt.ioevs[1].iv = 2,
+					.tgt.ioevs[2].uiv = vobj->owner->msc,
+				});
+		}
+	}
+}
+
 static void internal_yield()
 {
 	arcan_event_poll_sources(arcan_event_defaultctx(), conductor.timestep);
@@ -553,6 +571,12 @@ static uint64_t postframe_synch(uint64_t next)
 	case SYNCH_IMMEDIATE:
 	break;
 	}
+
+/* this is not the 'correct' time to do this for multiscreen settings, we would
+ * need to let the platform expose that event per screen and bias to the one (if
+ * any) the frameserver is actually mapped to the vblank on. */
+	forward_vblank();
+
 	conductor.in_frame = false;
 	return next;
 }

src/engine/arcan_event.c

@@ -409,12 +409,25 @@ int arcan_event_queuetransfer(arcan_evctx* dstqueue, arcan_evctx* srcqueue,
 
 /* for autoclocking, only one-fire events are forwarded if flag has been set */
 				case EVENT_EXTERNAL_CLOCKREQ:
-					if (tgt->flags.autoclock && !inev.ext.clock.once){
+					if (inev.ext.clock.dynamic == 1){
+						if (inev.ext.clock.rate){
+							tgt->clock.present = inev.ext.clock.rate;
+							tgt->clock.msc_feedback = true;
+						}
+						else
+							tgt->clock.msc_feedback = !tgt->clock.msc_feedback;
+					}
+					else if (inev.ext.clock.dynamic == 2){
+						tgt->clock.vblank = !tgt->clock.vblank;
+					}
+					else if (tgt->flags.autoclock){
+						tgt->clock.once = inev.ext.clock.once;
 						tgt->clock.frame = inev.ext.clock.dynamic;
 						tgt->clock.left = tgt->clock.start = inev.ext.clock.rate;
-						wake = true;
-						continue;
+						tgt->clock.id = inev.ext.clock.id;
+						tgt->clock.once = inev.ext.clock.once;
 					}
+					continue;
 				break;
 
 				case EVENT_EXTERNAL_REGISTER:

src/engine/arcan_frameserver.c

@@ -86,8 +86,14 @@ static void autoclock_frame(arcan_frameserver* tgt)
 			.category = EVENT_TARGET,
 			.tgt.kind = TARGET_COMMAND_STEPFRAME,
 			.tgt.ioevs[0].iv = delta / tgt->clock.start,
-			.tgt.ioevs[1].iv = 1
+			.tgt.ioevs[1].uiv = tgt->clock.id
 		};
+
+/* don't re-arm if it is a one-off */
+		if (tgt->clock.once){
+			tgt->clock.left = 0;
+		}
+
 		platform_fsrv_pushevent(tgt, &ev);
 	}
 	else
@@ -683,12 +689,15 @@ int arcan_frameserver_releaselock(struct arcan_frameserver* tgt)
 	atomic_store_explicit(&tgt->shm.ptr->vready, 0, memory_order_release);
 	arcan_sem_post( tgt->vsync );
 		if (tgt->desc.hints & SHMIF_RHINT_VSIGNAL_EV){
+			arcan_vobject* vobj = arcan_video_getobject(tgt->vid);
+
 			TRACE_MARK_ONESHOT("frameserver", "signal", TRACE_SYS_DEFAULT, tgt->vid, 0, "");
 			platform_fsrv_pushevent(tgt, &(struct arcan_event){
 				.category = EVENT_TARGET,
 				.tgt.kind = TARGET_COMMAND_STEPFRAME,
 				.tgt.ioevs[0].iv = 1,
-				.tgt.ioevs[1].iv = 0
+				.tgt.ioevs[1].iv = 0,
+				.tgt.ioevs[2].uiv = vobj ? vobj->owner->msc : 0
 			});
 		}
 
@@ -801,11 +810,36 @@ enum arcan_ffunc_rv arcan_frameserver_vdirect FFUNC_HEAD
 			goto no_out;
 		}
 
-/* for tighter latency management, here is where the estimated next
- * synch deadline for any output it is used on could/should be set,
- * though it feeds back into the need of the conductor- refactor */
+/* TIMING/PRESENT:
+ *     for tighter latency management, here is where the estimated next synch
+ *     deadline for any output it is used on could/should be set, though it
+ *     feeds back into the need of the conductor- refactor */
 		dst_store->vinf.text.vpts = shmpage->vpts;
 
+/*     if there's a clock for being triggered in order to be able to submit
+ *     contents at a specific MSC on best effort, forward that now. */
+		if (tgt->clock.msc_feedback){
+			struct arcan_event ev = {
+				.category = EVENT_TARGET,
+				.tgt.kind = TARGET_COMMAND_STEPFRAME,
+				.tgt.ioevs[0].iv = 1,
+				.tgt.ioevs[1].iv = 1,
+				.tgt.ioevs[2].uiv = vobj->owner->msc
+			};
+
+			if (tgt->clock.present &&
+				(tgt->clock.present + 1 <= vobj->owner->msc)){
+				TRACE_MARK_ONESHOT("frameserver", "present-msc", TRACE_SYS_DEFAULT, tgt->cookie, tgt->clock.present, "");
+				platform_fsrv_pushevent(tgt, &ev);
+				tgt->clock.present = 0;
+				tgt->clock.msc_feedback = false;
+			}
+			else if (!tgt->clock.present && tgt->clock.last_msc != vobj->owner->msc){
+				platform_fsrv_pushevent(tgt, &ev);
+				tgt->clock.last_msc = vobj->owner->msc;
+			}
+		}
+
 /* for some connections, we want additional statistics */
 		if (tgt->desc.callback_framestate)
 			emit_deliveredframe(tgt, shmpage->vpts, tgt->desc.framecount);
@@ -824,7 +858,8 @@ enum arcan_ffunc_rv arcan_frameserver_vdirect FFUNC_HEAD
 					.category = EVENT_TARGET,
 					.tgt.kind = TARGET_COMMAND_STEPFRAME,
 					.tgt.ioevs[0].iv = 1,
-					.tgt.ioevs[1].iv = 0
+					.tgt.ioevs[1].iv = 0,
+					.tgt.ioevs[2].uiv = vobj ? vobj->owner->msc : 0
 				});
 			}
 		}
@@ -1469,7 +1504,7 @@ bool arcan_frameserver_tick_control(
 				.category = EVENT_TARGET,
 				.tgt.kind = TARGET_COMMAND_STEPFRAME,
 				.tgt.ioevs[0].iv = 1,
-				.tgt.ioevs[1].iv = 1
+				.tgt.ioevs[1].iv = src->clock.id
 			});
 		}
 	}

src/engine/arcan_frameserver.h

@@ -184,7 +184,13 @@ struct arcan_frameserver {
 		uint32_t left;
 		uint32_t start;
 		int64_t frametime;
+		uint32_t id;
+		uint32_t present;
+		uint32_t last_msc;
+		bool msc_feedback;
 		bool frame;
+		bool once;
+		bool vblank;
 	} clock;
 
 /* for monitoring hooks, 0 entry terminates. */

src/engine/arcan_lua.c

@@ -4417,16 +4417,10 @@ bool arcan_lua_pushevent(lua_State* ctx, arcan_event* ev)
 				return true;
 			}
 		break;
+/* this is handled / managed through _event.c, _conductor.c and _frameserver.c */
 		case EVENT_EXTERNAL_CLOCKREQ:
-/* check frameserver flags and see if we are set to autoclock, then only
- * forward the once events and have others just update the frameserver
- * statetable */
-			tblstr(ctx, "kind", "clock", top);
-			tblbool(ctx, "dynamic", ev->ext.clock.dynamic, top);
-			tblbool(ctx, "once", ev->ext.clock.once, top);
-			tblnum(ctx, "value", ev->ext.clock.rate, top);
-			if (ev->ext.clock.once)
-				tblnum(ctx, "id", ev->ext.clock.id, top);
+				lua_settop(ctx, reset);
+				return true;
 		break;
 		case EVENT_EXTERNAL_CONTENT:
 			tblstr(ctx, "kind", "content_state", top);

src/engine/arcan_video.c

@@ -5212,6 +5212,7 @@ static size_t process_rendertarget(
 		arcan_vobject_litem* tmp_cur = tgt->first;
 		tgt->first = tgt->link->first;
 		tgt->link = NULL;
+		size_t old_msc = tgt->msc;
 
 		pc += process_rendertarget(tgt, fract, false);
 		nest = pc > 0;
@@ -5221,6 +5222,7 @@ static size_t process_rendertarget(
 
 		tgt->dirtyc += tgt->link->dirtyc;
 		tgt->transfc += tgt->link->transfc;
+		tgt->msc = old_msc;
 	}
 
 	current = tgt->first;
@@ -5236,6 +5238,7 @@ static size_t process_rendertarget(
 		return 0;
 
 	tgt->uploadc = 0;
+	tgt->msc++;
 
 /* this does not really swap the stores unless they are actually different, it
  * is cheaper to do it here than shareglstore as the search for vobj to rtgt is

src/engine/arcan_videoint.h

@@ -53,6 +53,9 @@ struct rendertarget {
  * compare to this and see if it is different */
 	uint64_t frame_cookie;
 
+/* media-stream count, increments on each update */
+	uint64_t msc;
+
 /* useful for link targets, ignore whatever shader is assigned and use shid */
 	bool force_shid;
 

src/shmif/arcan_shmif_event.h

@@ -292,19 +292,39 @@ enum ARCAN_TARGET_COMMAND {
 	TARGET_COMMAND_EXIT = 1,
 
 /*
- * Hints regarding how the underlying client should treat
- * rendering and video synchronization.
+ * Hints regarding how the underlying client should treat rendering and video
+ * synchronization.
+ *
  * ioevs[0].iv maps to TARGET_SKIP_*
  */
 	TARGET_COMMAND_FRAMESKIP,
 
 /*
+ * [AGGREGATE]
+ *
+ * STEPFRAME is a hint that new contents should be produced and synched and is
+ * influenced by any previously set FRAMESKIP modes, as well as if frame event
+ * feedback is set (SHMIF_RHINT_VSIGNAL_EV) or any custom timer sources has
+ * been requested for clocking.
+ *
+ * ioevs[1].uiv is the identifier of the step request source.
+ * For a custom CLOCKREQ this will match the ID provided in the source with
+ * a recommended range of 10..UINT32_MAX.
+ *
+ * Reserved IDs are:
+ * 0 : [rhint_vsignal],
+ * 1 : [present-feedback, see CLOCKREQ]
+ * 2 : [vblank-feedback, see CLOCKREQ]
+ *
+ * ioevs[0].iv represents the number of frames to skip forward or backwards.
+ *
  * in case of TARGET_SKIP_STEP, this can be used to specify
- * a relative amount of frames to process or rollback
+ * a relative amount of frames to process or rollback.
  * ioevs[0].iv represents the number of frames,
- * ioevs[1].iv can contain an ID (see CLOCKREQ)
- * ioevs[2].uiv (on CLOCKREQ) 0 or seconds (NTP- jan 1900 64-bit format)
- * ioevs[3].uiv (on CLOCKREQ) fractional second ( - " - ) in GEOHINT- tz
+ * ioevs[1].iv may contain a user ID or a reserved one (see CLOCKREQ).
+ * ioevs[2].uiv may contain the current attachment MSC (if avaiable)
+ *
+ * For present-feed
  */
 	TARGET_COMMAND_STEPFRAME,
 
@@ -1335,17 +1355,43 @@ enum ARCAN_TARGET_SKIPMODE {
 			uint32_t type;
 		} stateinf;
 
-/* Used with the CLOCKREQ event for hinting how the server should provide
- * STEPFRAME events. if once is set, it is interpreted as a hint to register as
- * a separate / independent timer.
- * (once) - & !0 > 0, fire once or use as periodic timer
- * (rate) - if once is set, relative to last tick otherwise every n ticks.
- * (id)   - caller specified ID that will be used in stepframe reply
- * (dynamic) - set to !0 if it should be hooked to video frame delivery rather
- *             than an approximate monotonic clock
- *
- * there is one automatic clock- slot per connection, and will always have
- * ID 1 in the reply.
+/*
+ * Used with the CLOCKREQ event for hinting how the server should provide
+ * autoclocked STEPFRAME events.
+ *
+ * There is >one< server managed coarse grained (25Hz tick) custom autoclock
+ * (dynamic = 0), any subsequent CLOCKREQs will override the previous setting.
+ *
+ * If (once) is set, it will not be re-armed after firing and (rate) represents
+ * the number of ticks that should elapse before firing. Otherwise the timer
+ * will be re-armed after firing.
+ *
+ * The (id) Will be provided in the returned stepframe,
+ *          with values 0 .. 10 reserved for other stepframe uses.
+ *          This matters only if you need to differentiate between different
+ *          kinds of stepframe requests.
+ *
+ * If (dynamic) is set to 1, the clock will be attached to presentation
+ *              feedback. If rate is set the STEPFRAME will fire once when
+ *              a new frame would be needed to be submitted to hit that
+ *              specific MSC. This will only fire once.
+ *
+ *              If rate is not set, each MSC increment will yield an event
+ *              until it is disabled with another CLOCKREQ.
+ *
+ *              See STEPFRAME for more information.
+ *
+ * If (dynamic) is set to 2, STEPFRAMEs will be emitted on each vblank of the
+ *              sink the segment is primarily mapped to. This does not have to
+ *              match any previous received OUTPUTHINT.
+ *
+ * The vblank dynamic clock act as a toggle, repeating the same CLOCKREQ would
+ * disable the previous.
+ *
+ * Being subscribed to a dynamic clock should be handled with care as it is
+ * very easy to drag behing in your processing loop and saturate the inbound
+ * queue. In such a case the dequeueing function might AGGREGATE
+ * (merge/discard) stepframe events.
  */
 		struct{
 			uint32_t rate;