Docker-less setup

[olivierlambert]

Improvement Description

Sometimes, you might want to use the software in a regular VM (or small host, like a Raspberry Pi) without relying on Docker, especially outside a purely dev oriented environment.

Rationale

From my personal point of view, I'm not using Docker nor want to manage containers in production. Doing a git pull and npm build is often enough to start in production mode and serve all of it via a reverse proxy (eg Nginx+LE). This also help to contain the dependency creep by knowing exactly the services you need and self-host them the way you want.

Proposed Solution

Nothing specific is required outside to document how to achieve it in the official doc or README file. I'm open to participate in that direction if it's OK for the project :)

Alternatives (optional)

No response

Additional Context

No response

Please check the boxes that apply to this improvement suggestion.

• I have searched the existing issues and improvement suggestions to avoid duplication.
• I have provided a clear description of the improvement being suggested.
• I have explained the rationale behind this improvement.
• I have included any relevant technical details or design suggestions.
• I understand that this is a suggestion and that there is no guarantee of implementation.

[rotimi-best]

@olivierlambert Thanks for opening this issue 😃.

Our stack is just Sveltekit and Supabase, the Sveltekit part is just like any frontend framework where you build and serve, that is straight forward.

However for the Supabase part, I am not sure how people want to do that part. There are 2 options according to my level of understanding now:

1. Self-host Supabase: There are a ton of resources on how to do that, the official docs is here. Once you self host, you will then need to connect this project to your supabase account in order to run the migrations to make sure you project matches exactly what is needed for the application.

2. Use Supabase cloud: With supabase cloud, you will also need to link it to the project so you can run migrations and deploy any change made to the repo, directly to your project.

For any of the 2 options, you still need to get these environment variables

   PUBLIC_SUPABASE_PROJECT_REF=<PROJECT REF>
   PUBLIC_SUPABASE_URL=<API URL>
   PUBLIC_SUPABASE_ANON_KEY=<anon key>
   PUBLIC_SUPABASE_BUCKET_URL=<Inbucket URL>

In addition, I need to write a script to apply changes in the repo to any supabase account.

Which of the 2 options matches your requirement?

[olivierlambert]

Hey @rotimi-best

First, (and I forgot to say that in my initial message): thank you for this great software.

Then, thank you for your very fast and detailed answer. You rock 👍

I have to admit I'm not used to Sveltekit and Supabase, so I probably need to learn a bit how it works and how I could build those.

Anyway, I'm all in for the self hosted Supabase, because I'm trying to self-host everything at my company.

[rotimi-best]

Thank you very much @olivierlambert. I am super happy to help you on your self hosting journey.

TBH I haven't selfhosted supabase before but I think the process should be well documented and smooth.

Do you mind sharing your use-case for classroomio 😃?

We can start taking baby steps to help you selfhost the whole stack, this is also gonna be helpful because we can document the selfhosting process in our documentation

[olivierlambert]

Sure, the use case is pretty simple: we'd like to offer a training/certification program on our open source virtualization stack (see https://vates.tech for the details). We started to have some courses on slides, but having a solution to deliver the courses with a quiz and such will offer a nice experience.

Obviously, this means we'll probably know to interconnect many software together (or to have an SSO for our existing customer portal to make them connect to all our apps seamlessly) but this is not the priority right now.

So I'm exploring all the different Open Source LMS platforms, I don't need something super complicated, I'm not a university or something, it's few courses (probably 10 at max on the long run).

[rotimi-best]

Oh this makes sense. This was my exact problem with the existing open source solutions. The stack was either complicated to setup and uses some stale technologies or the feature sets were more best suited for enterprises.

Do you want to try out the platform first before investing the time in selfhosting?

I can give you a quick demo or give you access to the platform to try out the cloud version yourself - whichever you prefer 😃

[olivierlambert]

Hmm why not, but usually I'm first installing it on prem and testing it. I'm crazy busy but if I can't attend the demo, someone else from my team could attend. I'll discuss this internally and book a demo via your cal link.

[rotimi-best]

Sounds good. That's fine. Whichever is fine by you. Please feel free to continue the convo here or just shoot me an email best@classroomio.com

[olivierlambert]

Thanks, we'll focus on the tech part in here, and have a convo in parallel with the relevant people at Vates. Thank you again for your great software and also your time to answer!

[olivierlambert]

So I'm back and now starting to actively try to install the stack locally, without Docker :)

If I understand correctly, the app I want to use is the "dashboard" one. I need to build that app and then to connect it to Supabase (it's another topic for now).

However, if I go in the dashboard folder and try to do npm i, I'm already stuck with:

$ npm i
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:*

npm ERR! A complete log of this run can be found in:
npm ERR!     /var/www/.npm/_logs/2024-01-18T12_06_42_206Z-debug-0.log

I'm using Node 18.

[rotimi-best]

Welcome back 😀. Not sure if I need to improve the communication in the README however to install the packages we use pnpm.

That means you should run pnpm i

[olivierlambert]

So… progress! \o/

Now I'm in front of this:

[olivierlambert]

So in short, I'm loading Nginx in front of npm run preview on a domain name, and when I'm loading it, I'm redirected to https://app.classroomio.com/404?type=org

[olivierlambert]

Okay so it's a bit better with npm run dev vs preview: now I have the main page. Do I have pre existing/default credentials?

If not, trying to register fails with:

[rotimi-best]

Oh, glad it worked. Yeah npm run dev is the command to get the frontend running.

@cannarron is actually working on a contributibution that adds dummy data into the db after running migrations. This gives you test users and courses to start interacting with.

You can't register (NetworkError when attempting to fetch resource) because you don't have the database layer up and running. Can you follow the guide in the README to setup supabase locally and you can add that to your environment variables.

With this, registration should go smoothly.

[olivierlambert]

I did install/deploy supabase (on another machine where I can run Docker), so it's running and I can reach it. But maybe you have a script somewhere on your own doing something I didn't?

edit: ah, I need to open some ports in the firewall.

[olivierlambert]

So I'm not sure exactly about the configuration between Supabase and Dashboard app. I followed the Supabase self-hosted guide and configured to listen on an IP that can be reach now from the VM running ClassroomIO.

But it's a bit confusing about which service/port I should use.

For example, on Classroom, I have:

• PUBLIC_SUPABASE_URL : I don't know which variable it matches on Supabase side. Is it SUPABASE_PUBLIC_URL on supabase side?
• PUBLIC_SUPABASE_BUCKET_URL: no similar variable on Supabase side.

edit: or are you accessing everything via Kong? Then it would be KONG_HTTP_PORT

[olivierlambert]

Ah I think I start to understand the issue. Dashboard app is behind an HTTPS frontend (Nginx doing SSL termination) and Supabase is on HTTP. Mixed content is blocked by the browser.

However in HTTPS, I only have self signed certificates for Superbase, and it seems to block the request from Dashboard app.

I wonder if it's possible to tell Dashboard app to ignore self-signed certs.

[rotimi-best]

Hi @olivierlambert, will try to answer all your questions.

Supabase environment variables

1. PUBLIC_SUPABASE_PROJECT_REF
   

2. PUBLIC_SUPABASE_URL and PUBLIC_SUPABASE_ANON_KEY

3. PUBLIC_SUPABASE_BUCKET_URL: I just realised it isn't used at all so I just created a PR to completely remove it.
   refactor: remove unused supabase storage path #129

[rotimi-best]

I wonder if it's possible to tell Dashboard app to ignore self-signed certs.

@olivierlambert I am not sure tbh. I haven't worked with self signed certificates. It seems to me that HTTPS/HTTP issue is a browser constraint and not one that can be controlled from a web app

[olivierlambert]

Okay so I disabled mixed content protection in the browser for now, at least I can move forward (I'll see later on how to fix this, but it's important to understand it's harder to self-host when your components aren't placed on the same machine).

Anyway, now I have a different problem (visible in the dev console):

1. GET http://xxxx:8000/rest/v1/profile?select=*&id=eq.undefined is failing with a 401 unauthorized error.
2. If I try to signup, I have http://xxxx:8000/auth/v1/signup request that is failing with a 500 Internal Server error.

What's also weird, is that I don't have any existing project in Supabase. Should I create one manually? Did I miss anything in there?

[rotimi-best]

Oh damn, so sorry I missed this notification. I guess moving this from an issue to a discussion muted the conversation for me some how.

I actually came back here because I thought you haven't replied me since then 😄. I will try to keep an eye on our chat here more often.

Now back to the issue you mentioned.

Yes please you should create a supabase project (in your selfhosted supabase), call it whatever you want. Then you will need to run the migrations (supabase/migration) against your project. I will give you a couple of steps to go over to get your project similar with what we have.

1. Link the codebase to your supabase project.

I am assuming for you to get the dashboard running, you have cloned the repo and you have already installed all dependencies.

• First you should go to the root of your project and push the db to your selfhosted project like this

pnpm supabase db push --db-url "postgres://user:pass@127.0.0.1:5432/postgres"

docs here

This should run the migration files on your newly created project on your hosted supabase.

2. Add your supabase credentials to the application.

With this new project you created and assuming linking your project went successfully. I remember we had a conversation earlier about where to get the right supabase credentials for the .env file. Please make sure you have the right creds in the .env file of the dashboard in order for the signup to be successfull.

Please let me know if you run into any issues, I am glad we are having this discussion, we will eventually write a documentation from your experience. 🫱🏾‍🫲🏻🫡

[olivierlambert]

Okay so now I have to find how to create a project in my selfhosted supabase first :D On it, thanks!

[rotimi-best]

Yeah 😀. That should be on the main supabase dashboard

[olivierlambert]

Well, not in the self-hosted version. Probably need to use the CLI to do it.

[olivierlambert]

I think I will abandon the self-hosted route for now: it's far far from being usable in a reasonable amount of time:

• Self hosted supabase is clearly not great, as this version doesn't provide everything needed to make the setup easily
• Even when decided to rely on a public/cloud version of Supabase project (from their website), I discovered many other surprises, like:
  • when doing the registration, I have no message to tell me it succeeded, so I'm still in front of the account registration like nothing happened, despite I immediately received an email
  • the confirmation link was broken and was using a redirect (from Supabase website) ending with localhost:3000
  • after replacing it, I finally managed to access it, but I had to answer a form on when/why I made the choice of ClassroomIO
  • all pages are taking ages to load

I will switch to the Cloud version even if I wanted to avoid it in the first place, but it's too young to be ready to do it. I understand how convenient tools like Supabase can be to develop fast, but they are a real pain in the ass to deal with when you need to self-host. I suppose you might need extra feature vs a regular Postgres database, but there is consequences on the "usability" for the average sysadmin.

[olivierlambert]

That would be really useful. In the meantime, we are in a rush here for our training program, so we'll go for the SaaS version. I'm just afraid about migrating back to the self hosted one later 😬 But we'll see.

[rotimi-best]

When you are ready to migrate, I can write you a script that dumps only your data and we can just import it into your self hosted DB.

What's the timeline of your training program?

[olivierlambert]

We'd like to play with the platform to create the courses in the following days. We can always provide it as a "beta" until we migrate later before going official, but we really need to get started in the next days.

[rotimi-best]

Sounds good. Mean while I will create a documentation on how to use the supabase cloud and connect it to a hosted classroomio dashboard.

[rotimi-best]

One point I noticed you mentioned which I didn't address was

all pages are taking ages to load

Yes, this only occurs when you run pnpm dev. Vite make the start time so fast, however when moving between routes, it begins to perform caching on each route making route navigation extremely slow. Ideally you should run the following when selfhosting our frontend:

$ cd apps/dashboard && pnpm run build && pnpm run preview

This would be a lot faster because all the routes are prerendered at build time.

[rotimi-best]

@olivierlambert Thanks for your patience, I just completed the video tutorial showing how to setup your supabase project in their cloud offering and just connect to our self hosted (or local) frontend.

Here is the video: https://www.youtube.com/watch?v=dAZAZ0oiky0

I am also working on adding this to our documentation. However in order not to delay getting it to you, I decided to share the video with you first and once I am done with the docs (practically the same thing in the video), I will also share it with you as well.

btw how is your course creation coming along 😀?

[rotimi-best]

@olivierlambert In addition to the video, I have written a documentation on how to setup a Supabase Cloud Project and link with our application.

With this approach, you can selfhost our frontend application, and use supabase cloud to manage all the data and the backend which you will have full control over.

https://www.classroomio.com/docs/contributor-guides/supabase-cloud