I Can't Add New Timelines To Ghosts Through API

[danross826]

A coworker of mine set up the Ghosts API while I've been working on the front end of our app. I can add new Machines through the API post request as well as retrieve them with a get request, but I can't add new timelines through Ghosts timeline post request. I don't get an error message, and it seems to work when I send the request. However, when I use the get request to pull down all the timelines in a machine, I just get an empty object. Nothing appears to be added despite the lack of an error message. Can anyone help me with this?

[sei-dupdyke]

Hi, thanks for checking out the project.

Hmm, it might be a couple of things — let's start with what you're posting as a timeline. Would you please post the json payload for POST to /api/timelines?

[danross826]

Yes.

[danross826]

I used wireshark to analyze the traffic between the vm with the client and the api server. It seems like they're communicating, but when the api server communicates back with the client, it gets a 404 error. I don't know if that means anything to you. Like, I'm seeing the api server trying to communicate back to the client with the client updates api address, and it throws a 404.

[sei-dupdyke]

Note: If you are trying to use those client endpoints they will fail, the client uses some custom headers to identify themselves.

404 is expected for some endpoints - like if there is no client update currently, then a 404 is expected behavior.

[danross826]

Well, I'm not sure if I'm misunderstanding something with order of operations, but I've submitted timelines for this machine with the post api addresses and they just stay in machine updates with a status of 0. I thought when the client communicated with the api server that they'd be processed and my timelines would show up in the timelines table. I don't know if the server communicating back with the client and getting a 404 might be related to that not working how it's supposed to.

[danross826]

I think it's possible that the return messaging that's getting the 404 error is hitting a port that the client isn't listening to. Wireshark says it's hitting port 52550 while when I check my windows machine with the client, it says it's listening on port 135. I'm looking to see if there's a way to change the port on either the client or the api server. I didn't set up the api server so I'm just going through the files to see if there's something that might correspond to this.

[sei-dupdyke]

Well, I'm not sure if I'm misunderstanding something with order of operations, but I've submitted timelines for this machine with the post api addresses and they just stay in machine updates with a status of 0. I thought when the client communicated with the api server that they'd be processed and my timelines would show up in the timelines table. I don't know if the server communicating back with the client and getting a 404 might be related to that not working how it's supposed to.

Hmm, I think you have it right:

• client checks in via web request with information about itself — including its globally unique MachineId, but there are no updates for that MachineId, so client receives 404 status code from the server. Client understands this as "no updates found for me".
• you submit a machine timeline for a particular MachineId via the API.
• the server holds it in machine_updates table until that client checks in (basically asking, "do you have anything new for me to do?)"
• client checks in again, that update you posted is found by matching the MachineId of the incoming client request to the machine you indicated that the update was for, and the response code is now 200, which tells the client "here is a new thing for you to do".
• server marks that update as processed ("-9" IIRC).

I think it's possible that the return messaging that's getting the 404 error is hitting a port that the client isn't listening to. Wireshark says it's hitting port 52550 while when I check my windows machine with the client, it says it's listening on port 135. I'm looking to see if there's a way to change the port on either the client or the api server. I didn't set up the api server so I'm just going through the files to see if there's something that might correspond to this.

What you are seeing in wireshark is that the client makes a request to the server on 52550 and the latter returns 404. The client receives this on an ephemeral port (135 in this case), like this:

client --> server:52550 -> responds with "404" --> received on client:135.

---

couple of things to check:

• what is the uuid in ./instance/id.json? (this is the global Id for this client)
• if you do a select * From machine_updates; does the machineId match the above?

[danross826]

Okay, so you guys recently updated ghosts so I reinstalled docker and the new version of ghosts api and all that. I also downloaded a new client on my other machine that I'm trying to run ghosts on. Okay, so I get that when you activate a new client it creates an id.json file that has that client's new machine id, and it adds that id to the machines table in the ghosts api server. So, I should be able to create new timelines with that machine id and it should be able to run and follow those timelines. I've attempted to do this on swagger with the example timeline you have on the timeline post api address, and now I'm just getting a 500 error every time.

[sei-dupdyke]

My apologies, I finally realized this morning what the issue is that you are pointing out!

For POST /api/machineupdates the previously generated sample json payload was:

{
  "machineId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "type": "Timeline",
  "activeUtc": "2024-03-23T13:39:44.558Z",
  "status": "Active",
  "update": {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "status": "Run",
    "timeLineHandlers": [
      {
        "handlerType": "BrowserFirefox",
        "initial": "string",
        "utcTimeOn": {
          "ticks": 0
        },
        "utcTimeOff": {
          "ticks": 0
        },
        "utcTimeBlocks": [
          {
            "ticks": 0
          }
        ],
        "handlerArgs": {
          "additionalProp1": "string",
          "additionalProp2": "string",
          "additionalProp3": "string"
        },
        "loop": true,
        "timeLineEvents": [
          {
            "trackableId": "string",
            "command": "string",
            "commandArgs": [
              "string"
            ],
            "delayAfter": "string",
            "delayBefore": "string"
          }
        ],
        "scheduleType": "Other",
        "schedule": "string"
      }
    ]
  }
}

Any of the settings with "ticks" are timespans types, and they don't serialize properly for our purposes here in this method. I fixed this by explicitly creating a sample object that gets displayed now with version 8.0.7 of the container:

{
  "machineId": "8da92fc7-01af-4261-80e0-33539a7f251a",
  "type": "TimelinePartial",
  "activeUtc": "2024-03-23T14:14:37.411893Z",
  "status": "Active",
  "update": {
    "id": "102d599b-df89-4ed5-891f-e0a09f14210e",
    "status": "Run",
    "timeLineHandlers": [
      {
        "handlerType": "BrowserFirefox",
        "initial": "https://cmu.edu",
        "utcTimeOn": "00:00:00",
        "utcTimeOff": "23:59:59",
        "utcTimeBlocks": null,
        "handlerArgs": {},
        "loop": true,
        "timeLineEvents": [
          {
            "trackableId": null,
            "command": "browse",
            "commandArgs": [
              "https://sei.cmu.edu"
            ],
            "delayAfter": 30000,
            "delayBefore": 0
          }
        ],
        "scheduleType": "Other",
        "schedule": null
      }
    ]
  }
}

Sorry for the confusion, I will try to be sure to check these swagger functions more closely with future updates.