Pagination: ACSS

[dragontheory]

Show/hide based on dividing total number of visible search result rows with total search results returned.

Note: search result rows will have options to view as cards and will be responsive. In that case, we may need to go to an infinite scroll option via CSS media queries.

[bob2517]

Will start off with working with page numbers. Infinite scroll would follow on from getting that working.

[bob2517]

Some thoughts on how to do this.

There are 3 ways to go about this, each with pros and cons.

Currently, using the existing API, or a superhero JSON copy, the only way the pagination spec would work is if we loaded up ALL the records into memory, because there are no paging options on the API at all.

Obviously this isn't very practical.

Ideally we would want to send the API back-end this information to fetch data:

starting record ID
ending record ID

But that would only work if the ID was a number.

If the ID was a unique string, we would need to send over this data:

page number
starting record ID
total number of records that we need for the current page display

That would be ideal. However, most projects would need to set up a back-end API that could handle this. That may or may not be acceptable as a solution, as it isn't a regular thing to do - this is new ground and people don't generally do this sort of JS pagination thing, so they wouldn't have built it into their API.

The other option is complicated to set up from an ACSS point of view, but this would work with regular APIs that have a regular fixed paging facility, which should be most real APIs.

I'm going to lay out that in a sec, once I've got my head around it.

[bob2517]

The variables we would need to keep track of:

page number (integer)
number of rows visible (integer)
total number of pages (integer)
number of records per page that the API can handle (integer)
API page numbers required to get all the rows needed on the front-end - ie. we may need records from the bottom of page 2 & the top of page 3 of the API to fetch the records required for our display. So there could be multiple AJAX calls to fetch the data we need. So this variable would be an array. (array) (insert meme)

I think that's it. That would enable this system to work with any API with any paging facility. In theory at least. If it works.

[bob2517]

Next would be finding an API we can use that has a paging facility. Will look around. If not, I'll set one up. Would rather use something that is already available though.

[bob2517]

This method would also work with infinite scroll. There's no real difference in the technique - it's just a case of grabbing the next page contents (which may mean one or more calls to the API - same as the click paging).

[bob2517]

Had a look around and didn't find a decent API with paging that has content that matches what we need, like names and addresses, etc.

I think it's probably going to be easier if I just set up a REST point and split up the superhero JSON files and simulate paging on a server - that's going to be a lot quicker than setting up a database. I can put it somewhere on the internet, like on the ACSS docs site. It won't cost anything then either. We can get a valid CRUD flow then too, simulate errors, etc.

Can you email me the superhero JSON file to support@, or put it in the repo somewhere so I can grab it? Ta.

[bob2517]

If needed, I can set up a database for it later on when it looks like everything's working. Then it will be fully functioning CRUD.

[bob2517]

I'll use the existing app JSON meanwhile, just so I can test the endpoint. And then I'll slot in the superhero records when you send it.

[bob2517]

Great, thanks. I'll grab it from there.

[bob2517]

There's lots of cool data in that file - nice :)

[dragontheory]

Can you email me the superhero JSON file to support@, or put it in the repo somewhere so I can grab it? Ta.

Here is the address I just sent an invite to.
https://github.com/dragontheory/dragontheory/tree/main/data

Tutorial I read to do the host the JSON:
https://victorscholz.medium.com/hosting-a-json-api-on-github-pages-47b402f72603
They said if you do it right, you can do CRUD operations but I'm not sure I did it right... lol

This is where I generated the random names from
Fake data generator
https://omatsuri.app/fake-data-generator

Superhero JSON API. I just copied it from here. They also have the JSON broken up into categories and individual JSON files for each supe.
https://akabab.github.io/superhero-api/api/

Since this has a lot of detail for each supe (even multiple images for a carousel), I envisioned using this for designing the aside panel. There will be an [ edit ] button and suddenly all the information is editable.

I can do that by putting all the content into a form by default and styling it as if it normal non-editable data. Then when users click [ edit ], it is easy to change the styles so the data looks editable.

Same is true for the normal generated data.

Creating a basic form modeled after the supes data. Unless that is going to get in your way?

[bob2517]

Creating a basic form modeled after the supes data. Unless that is going to get in your way?

No, that's fine, go for it.

[bob2517]

I'm creating a database for the supers which can be queried and edited - it will save time later on.

[bob2517]

I've created the database structure - it's nothing fancy - just enough so we can replicate the API with paging. I'll populate it with data offline, set up the code for the API and push it live once its working. I have an admin area on my offline docs site where I can test it out.

[bob2517]

If you come across anything when making the forms, or have any data related questions, let me know.

[bob2517]

The REST endpoints will be something like:

Get a page of data (GET):
https://activecss.org/supers/?perPage=20&page=2

Fetch supe data with a single id (GET):
https://activecss.org/supers/?id=23

Save data from a form (with POST variables):
https://activecss.org/supers/update

Delete data from the database (with POST variables):
https://activecss.org/supers/delete

We may need to host the repo somewhere to get around CORS issues, but we'll see. I have a feeling that we'll need a proper web server to get the ajax stuff working. It's also worth putting it into a secure area, so only certain people can edit the data. I can host it on my server if you like and set up security for it, unless you have somewhere where it can be properly hosted and put behind a secure login.

[bob2517]

If you want to get a proper domain name for it, it could be pointed at my live servers and I can set up a real website behind an nginx admin login. It's not a big deal to do that.

[bob2517]

If we did that, we would need a new PRIVATE repo which would store the back-end code which would provide the API and CRUD back-end code, plus it would keep the actual database (with INSERT commands, which would create the database from scratch) so that it could be replicated on a different server if I suddenly got hit by a bus.

[bob2517]

My live server is super fast too, excuse the pun.

[dragontheory]

I have free accounts with Bluehost, Cloudflare and Netlify. Cloudflare has HTTP3. They all have CDNs. Could we leverage them?

[dragontheory]

I also have the domains https://D7460N.io/ and https://D7460N.dev/

Although https://D7460N.dev/ seems to be in an error state...

[bob2517]

I'll check and see if it's feasible. We do need full control of the back-end though. I'm just wrapping up the import of the supers JSON into a MySQL database and will take a look after that. If those server accounts won't cut it, I'll ask you to point your domain to my live server when I put the API live. Will let you know.

[dragontheory]

I'll check and see if it's feasible. We do need full control of the back-end though. I'm just wrapping up the import of the supers JSON into a MySQL database and will take a look after that. If those server accounts won't cut it, I'll ask you to point your domain to my live server when I put the API live. Will let you know.

I can give you access to those accounts if necessary. No problem at all.

[bob2517]

Cool

[bob2517]

I've put the JSON file into a database now, so the API won't take long to set up now. There are 563 superheroes there in total.

I had a look at the various options and I'm just going to set up the API via the live docs site. It will be fast and reliable and quicker for me to set up. I'll do it so that it mimics the existing supers API, with the addition of "page number" and "records per page" fields. I might put in an "order by" option in the API as well, just in case that is needed.

[dragontheory]

Excellent! Thank you!

[bob2517]

I've written the get supes API call offline and it's tested. I'll do the get supe by ID API call and then put it all live.

[dragontheory]

OK. How best to turn the right panel form elements to plain text without undoing your hard work?

Comment the input version and add a plain text version (although I don't know how the plain text version is supposed to look like)?

@each $fieldKey in $profileFields {
  $asideHTML: $asideHTML +
  "<list-item><list-cell>" + $apiSchema[$fieldKey].displayTitle + "</list-cell>" +
  "<list-cell>" + $apiSchema[$fieldKey].fieldType + escapeHTML(getVar($item, $apiSchema[$fieldKey].apiLocation)) + "</list-cell></list-item>";
  /* "<list-cell><input type='" + $apiSchema[$fieldKey].fieldType + "' id='{$fieldKey}' name='{$fieldKey}' value='" + escapeHTML(getVar($item, $apiSchema[$fieldKey].apiLocation)) + "' required /></list-cell></list-item>"; */
}

[dragontheory]

Most everything is either HTML layout AKA scaffolding or lists.

The only dynamic parts are the lists, though some of the lists drive the visibility of some of the scaffolding. Everything else is static and front-loaded per JAMstack architecture specs.

Would you componentize just the lists since they are the only parts that touch the back-end?

Since almost every list is structured the same (both horizontal and vertical, thanks to css grid), there could be just one list component with many iterations.

That would cut down on much of the redundant code - which is nice for other more experienced DEVs to see. Although, as you said before, I don't want to be TOO DRY and loose flexibility. But CSS can handle all the different various iteration layouts. The only major difference for each iteration would be the data itself, as far as I can tell.

Is that a good idea or a bad idea? Would componentizing and iterating be more or less performant, especially considering the infinite scroll feature?

Do you have an ACSS example of how to iterate a single component for multiple data calls?

[bob2517]

Componentizing would be less performant. Looping would happen in the component event componentOpen for a general list component. I don't see any particular benefit other than separating out the logic - you will still need to loop and output the same. Did you want to keep the list-item and list-cell names? Personally I don't see the point, though I can take a look later on after work if you like, but you will need to provide an example HTML structure so I can work with that if you want it changed. You may gain in having it separated, but you lose time in finding where the component is rendered and trying to find it - the code is never re-used too, so it won't get any drier.

[bob2517]

If you just want to separate out the HTML, you could always set up a separate custom event for drawing the HTML, and trigger it. That's another way of doing it that would be more performant than using a component.

[bob2517]

I want to set up proper ACSS functions too, which would also help separate out logic - but that's a core fix.

[bob2517-whiteline]

(It's me under my work account before work.) Am I right in thinking that you want to separate out the HTML so that it's easier to edit? And make the code clearer in that function?

What I would do is separate out that larger function into smaller custom events and trigger those. That would be the most performant way to do it and save having to change the HTML markup. I can take a look at that tonight if you like - it's a quick thing to do.

[dragontheory]

Am I right in thinking that you want to separate out the HTML so that it's easier to edit? And make the code clearer in that function?

Actually, no. I want to preserve the custom HTML tags and preserve the raw loading speed of the scaffolding on the page. The dynamic lists are what I was talking about above. Although being able to use a single list component multiple times, each time calling different data with ACSS would be very nice.

What I would do is separate out that larger function into smaller custom events and trigger those. That would be the most performant way to do it and save having to change the HTML markup. I can take a look at that tonight if you like - it's a quick thing to do.

Now that I think about it, the lists are (or will be) dynamic anyway. So, never mind. Guess I was going down a rabbit hole with that.

Although I have a delay of 1s that fades the app in after the DOM (scaffolding) has loaded. The dynamic content comes in AFTER that and doesn't fade in (just loads normally), which sort of defeats the purpose of the fade in. Is their a way to delay the app fading in until all the dynamic data has completely loaded (not just the scaffolding)?

Could it be as simple as reversing the load order so that everything fades in together 1s after loading?

domReady(() => {

  //  Load ACSS first?
  ActiveCSS.init({
    configLocation: '/assets/js/config.acss
  });

  //  Then display body when DOM is loaded
  document.body.style.opacity = 1.0;
  
});

[bob2517-whiteline]

Dynamic CSS is a feature of ACSS, so CSS commands work directly in ACSS. You can do all sorts of animation chaining easier than using static CSS, with ACSS using delayed action commands.

With that in mind, I would do this:

/* in body:preInit */
  $initialised: false;

/* in panel-list[data-search-results]:fetchRows */

@if (!$initialised) {
  body {
    opacity after 1s;
    $initialised: true;
  }
}

That means you are delaying the opacity CSS change for 1s;

I would put it somewhere after the ajax call in the fetchRows event. That way you are guaranteed to have everything loaded.

[dragontheory]

OK. Got the values displaying. Followed the quotes for value="".

Is this right?

"<list-cell>" + escapeHTML(getVar($item, $apiSchema[$fieldKey].apiLocation)) + "</list-cell>

[bob2517]

Whatever it was before, yes. Those functions are needed to work with the JSON schema system.

[bob2517]

You won't be able to have individual components for each item, if you are trying to do that. You can have a component for the outer list, but not for each individual item. ACSS doesn't yet support functions directly on component HTML, hence why I did it the way that I did to begin with.

[bob2517]

Which is why I said you could separate out the code into a separate event, and trigger that. That way the code remains as it was, it just gets moved into its own looping event.

Although being able to use a single list component multiple times, each time calling different data with ACSS would be very nice.

That's not possible with the existing component syntax. It will be, but I've not built anything in to allow conditional HTML content for components yet, and it's not a quick fix. Hence why I "render" the HTML content. It's not any worse than outputting it as a component. In fact, it's more performant to just render it. It just doesn't look as pretty.

[bob2517]

If you had a JSON schema that was unchanging - like at your work place - you can do it, ie. have a hard-coded item component, because you don't need those extra JS functions. But your UI is set up to work with variable JSON schemas - like different database structures - which is done simply by changing the values in the arrays at the top of the ACSS. But the sacrifice is that it's a bit more complicated when rendering the HTML for the row and aside. "getVar" is a new ACSS JS function that fetches the value of a variable based on a string which contains the name of the variable to fetch. It means here that you can have different row fields display in a row just by changing the JSON schema array items.

I do plan to change the syntax so that you don't need getVar or escapeHTML, but as I say, it's not a quick fix. But it will be made easier at some point to reference variables indirectly by a string containing the name of the variable. When that happens, it will be possible to use an item component. Until then though, rendering it like this is your only option. But it's not bad as it is - it's actually more performant to render like this. It's just got the apparency of having a "lower-level" feel to it.

[dragontheory]

You won't be able to have individual components for each item, if you are trying to do that. You can have a component for the outer list, but not for each individual item. ACSS doesn't yet support functions directly on component HTML, hence why I did it the way that I did to begin with.

Agreed. That would be overkill. Wasn't interested in componentizing each individual list-item. Just the outer list as you said.

[dragontheory]

It means here that you can have different row fields display in a row just by changing the JSON schema array items.

Oh wow. That is really cool. Thank you.

[dragontheory]

In fact, it's more performant to just render it. It just doesn't look as pretty.

Agreed.

[bob2517]

Ah, good. I've set up a ticket for the required syntax needed to do the individual items as a component, anyway, as it would make things easier sometimes. I did try to do the list item as a component to begin with, as it is a cleaner approach, but then realised it wasn't going to work and fell back to "old-school" rendering. I say old-school - it's only a year or so old.

[dragontheory]

OK.

Trying to get the images to show up on the intranet enterprise GitHub (no internet).

Uploaded all 746 sm images to the intranet enterprise GitHub (to the admin's chagrin, I am sure) and changed the call in the manual-database.js (line: 180) from this ...

'https://cdn.jsdelivr.net/gh/akabab/superhero-api@0.3.0/api/images/sm/1-a-bomb.jpg'

To the relative path (like the .svg logo), like this...

'/images/supes/sm/1-a-bomb.jpg'

And it is not working!

It is still calling the cdn.jsdelivr.net address...

What am I missing?

The directory structure is conventional, like this...

|_assets
   |_css
   |_images
   | |_supes
   |   |_sm
   |     |_1-a-bomb.jpg
   |_js
  

[bob2517]

Do not fret - you didn't waste your time. You'll need to clear your cache after any change to the data - the database is probably unchanged from the first time you ran it and is sitting in localstorage, so all your images are still pointing to the original CDN.

[bob2517]

Lines 7 to 31 in the manual-database code give the logic for that.

[bob2517]

This could replace that whole section if you want to temporarily comment that whole section out, so that fresh data is fetched on every page load.

var superRows = getRawRows();

[dragontheory]

This could replace that whole section if you want to temporarily comment that whole section out, so that fresh data is fetched on every page load.

var superRows = getRawRows();

Which section? Lines 7 to 31 in manual-database.js?

So replace lines 7 to 31 in manual-database.js with this?

var superRows = getRawRows();

[bob2517-whiteline]

Yes, but I would comment those lines out to save hunting for it later on.

[dragontheory]

Commented and it's working! Woohoo!

[bob2517]

Great! :)

[dragontheory]

Good morning!

Happy Monday!

New position today. Hope it goes well.

Wanted to change 'name' to 'alias' but when I do so in the config.acss it breaks the column in the search results. I don't see where else to change it in the apiLocation.

Get "undefined" in the 'Alias' column.

It was this...
config.acss line 13

 alias: { displayTitle: "Name", apiLocation: "name", fieldType: "text" },

Changed to this...
config.acss line 13

 alias: { displayTitle: "Alias", apiLocation: "alias", fieldType: "text" },

What am I missing?

Thanks!

[bob2517-whiteline]

Just a quick one, as I'm at work...

makeDataObj is the function you need to look at in the manual-database file. That has the JSON structure. If you look in there, you will see that "alias" is not defined on the same level as "name". The apiLocation property relates to that structure. The API structure matches the original API structure from the internet API.

[dragontheory]

Ah. OK. I will take a look at that. Thank you.

[dragontheory]

OK. Changed line manual-database.js 122 from this

name: thisRow.people_name,

To this...

alias: thisRow.people_name,

And it worked.

Thank you again.

[bob2517-whiteline]

Great :)