NO ISSUE, just question FILE DOWNLOAD ?

[MyRaspberry]

play with a ESP32-S3 N16R8
and setup FFAT 10MB partition.
and write dir and file (append 1 line / min )
/data/readings.csv

does THIS tool help me to DOWNLOAD that file via Browser?

the example FSBrowser i use, not even print that file (content)
my status : at GIT

THX for sharing your work

[cotestatnt]

Hi @MyRaspberry

Yes, it can.
Some time ago I've written a sketch to visualize a csv, edit and download
I just have to remember where I saved it....

However, it has little to do with the library in question, it's just HTML and Javascript coding.

[MyRaspberry]

i just worry about the file size can grow over the memory of the chip,
what also limits the html it can produce dynamically,
so how JS could serve big files?

[cotestatnt]

It depends on the size of your csv file.
Aniway, this library could serve web pages even from a SD memory card if a lot of space is needed.

Naturally, the larger the size of the csv file, the longer it will take to load the page that displays its content.

This is the result with my ESP32-S3 (only 1.3Mbyte for LittleFS filesystem) using my smartphone as Access Point.
Test with a file of approximately 110kb (1000 records)

and this instead with a file of approximately 1000kb (8000 records)

If you only need to download, the problem does not arise because it is not necessary to upload the contents of the file to the web page as in this case.
It will be sufficient to populate a list with the files to download (if there are more than one) and a button to start the download.

[MyRaspberry]

thanks,

anyhow i started coding the file thing a ? better ? way,
making daily files like "data/readings_2024-01-08.csv"
will see tomorrow if it works.

and then experiment with your lib

thanks for your time

[cotestatnt]

making daily files like "data/readings_2024-01-08.csv"

This is a great idea.
The download page at this point will only have to create a list based on the contents of the filesystem and allow you to download the files.

If you need, I have some working sketches that do similar things (not with csv, but basically it's the same behaviour)

[MyRaspberry]

-1- that first part seems to work:
find {"type":"file","name":"data/readings_2024-01-09.csv", "size": 21267 B } ( ! size after 8 hours already )

//update 2024.01.10:
[
{"type":"file","name":"data/readings.csv", "size": 2067 B },
{"type":"file","name":"data/readings_2024-01-08.csv", "size": 19537 B},
{"type":"file","name":"data/readings_2024-01-09.csv", "size": 61945 B },  <<< 62KB /d
{"type":"file","name":"data/readings_2024-01-10.csv", "size": 15032 B }
]

,rec,datetimes,A0,A1,A2,
,1785,2024-01-09 00:00:49,0.00,20.05,1.03,
,1789,2024-01-09 00:01:49,0.00,18.95,0.71,
,1793,2024-01-09 00:02:49,0.00,19.10,0.61,
...

tomorrow i know how big it will be.

-2- i can also now print the content using the server.streamFile
String contentType = getContentType(path);
File file = FILESYSTEM.open(path, "r");
server.streamFile(file, contentType);
file.close();

inside my added function to the FSBrowser code :
void handleFilePrint() { // on /print?file=/data/readings.csv

-3- BUT THAT IS NOT A DOWNLOAD ? HOW ?
i also failed to find something in your
esp-fs-webserver-master\src\esp-fs-webserver.cpp

It will be sufficient to populate a list with the files to download (if there are more than one) and a button to start the download.

sounds like you know how, but i have no idea what you refer to.

-4- i am lost,
while years ago i play

• esp2866 and SPIFFS
• and the /data/ dir in Arduino code and a
• Arduino IDE TOOLS menu entry UPLOAD

and after using Circuit Python just come back to Arduino IDE

now i see that is ?DISCONTINUED?

and having a ESP32-S3 N16R8 module i look into using FFAT
where i miss the way to handle any files there regarding UPLOAD & DOWNLOAD..
so finding FSBrowser i hoped that might be the way, but that is very incomplete.

also i miss to see anything ? regarding other tools
is there something from the Espressif tools?

or ? i have one of that boards with 2 USB C ports,
is there a code ?for Arduino IDE?
to use one port of that board as a USB STICK with file access to FFAT?
HA, that would be great

[MyRaspberry]

thanks anyhow,
i got the basics working.. see GIT

[cotestatnt]

Hi @MyRaspberry

I'm happy you already solved it.
In any case, if you want to take a look, I added an example taking inspiration from your needs

https://github.com/cotestatnt/esp-fs-webserver/tree/master/examples/csvLogger

[MyRaspberry]

yes, thanks, i love it, looks great and is very professional,

i see you use littleFS and the /data/ dir in the arduino project,
means what? did you use a upload tool?
because as i see that you create that log file in the ESP

• but after 'online edit' ... you can download it to your computer ( via that index.html download button )
• and possibly store it again in the arduino /data/ dir? for a new setup ?

i assume that i can not use it for my little FFAT exploring project?
but i could try it on that esp32-S3 N16R8? BUT NOT IDE 2.2.1 ? or need ESP32?
anyhow about your web page / css / js / i could learn a lot.
thanks again!

can you open that? my page snap

[cotestatnt]

Of course you can use it!
The library is compatible with any type of Espressif microcontroller including the ESP8266.

If you prefer to use FFat, simply include the "FFat.h" file and replace LittleFS with FFat wherever you see it in the sketch.

As for uploading the data folder, it is managed directly by the library using the included /setup page, Update & FS tab.

Once you have uploaded your sources, you can edit them directly using the included /edit page
Soon I will add a new version of /edit that allows you to download files, but in the meantime you can simply copy and paste to transfer any changes to your PC

Regarding your working HTML code, you can simply put it on a specific page: if you don't want write from beginning, open dev tools of your browser, move to the "source" tab and copy the content.
If the name page is index.htm or index.html it will be used as default page

#####Edit:#####
In order to use FFat some small chages are needed in the sources because FFat treats the / character in folder paths a little differently.
In the file assets/js/index.js edit the function as this listFiles() in order to remove the last '/' from url when a folder is passed to the function

// Fetch the list of files and fill the filelist
function listFiles() {
  var url = '/list?dir=' + dataFolder;
  if (url.charAt(url.length - 1) === '/')
    url = url.slice(0, -1); // Remove the last character
    
  fetch(url)    // Do the request
  .then(response => response.json())    // Parse the response
  .then(obj => {                        // DO something with response
    fileList.innerHTML = '';
    obj.forEach(function(entry, i) {
      addEntry(entry.name);
    });
    // Load first file
    loadCsv(dataFolder + obj[0].name);    
  });
}

[cotestatnt]

The print screen is from the twin library that uses the ESPAsyncWebServer library for the webserver instead of the one included in the core, but the result does not change (I develop them more or less in parallel)

From some tests I've done it seems to be faster and more responsive.

[MyRaspberry]

as expected nothing is easy....
first ( not yet reading above and with my possibly wrong thinking ) i try a extra working environment

• on a laptop ( win 10 )
• Arduino IDE 1.8.19
• installed lib esp-fs-webserver 1.2.8
• add download copy example from MASTER esp-fs-webserver-master\examples\csvLogger
• board ESP32 : NODE ESP-32S V1.1
• partition [default]
• press the RIGHT BUTTON for UPLOAD this sketch
  ( for first time i used erase all flash [enabled], who knows what i play last year ?CircuitPython? )

try set laptop wifi to
ESP_AP
password 123456789
from laptop
get:

Oops.

The site at http://www.msftconnecttest.com/redirect has experienced a network protocol violation that cannot be repaired.

The page you are trying to view cannot be shown because an error in the data transmission was detected.

Please contact the website owners to inform them of this problem.

try again to powerup via charger
try login via mobile
sign in to ESP_AP
the webpage at https://192.168.4.1/setup could not be loaded because
net::ERR_CONNECTION_REFUSED

[cotestatnt]

I'm sorry @MyRaspberry, but a pull request I've merged some times ago has introduced this bug due to a wrong reply when ESP is acting as captive portal ( "https:://" instead http://).

Once an ESP32 has succesfull connected to a SSID it store the credentials, so I've never realized before about this bug.

With last commit and releases I've fixed it.
I've added also some more features like manual WiFi configuration (fixed IP, gatway, subnet)

[MyRaspberry]

sorry, had to fail with a other idea first.....

// 12.1.2024 update to v 1.3.0 and take also that new example

Terminal show continuous booting

05:45:39.592 -> rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
05:45:39.639 -> configsip: 0, SPIWP:0xee
05:45:39.639 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
05:45:39.639 -> mode:DIO, clock div:1
05:45:39.639 -> load:0x3fff0030,len:1184
05:45:39.639 -> load:0x40078000,len:13260
05:45:39.639 -> load:0x40080400,len:3028
05:45:39.639 -> entry 0x400805e4
05:45:39.780 -> ERROR on mounting filesystem. It will be formmatted!
05:45:39.780 -> ets Jun  8 2016 00:22:57
05:45:39.780 -> 
05:45:39.780 -> rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
05:45:39.780 -> configsip: 0, SPIWP:0xee
05:45:39.827 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
05:45:39.827 -> mode:DIO, clock div:1
05:45:39.827 -> load:0x3fff0030,len:1184
05:45:39.827 -> load:0x40078000,len:13260
05:45:39.827 -> load:0x40080400,len:3028
05:45:39.827 -> entry 0x400805e4
05:45:48.649 -> ets Jun  8 2016 00:22:57
05:45:48.649 -> 

again, above i mentioned that i use partition DEFAULT ( whatever that means )

should i change that?

[cotestatnt]

You are trying to use a FFat based sketch selecting a SPIFFS based partition scheme (the default one),
Of course it will not work!

Also, up until now you've always said you have an ESP32-S3, but the board you selected uses a "normal" ESP32 so I'm guessing this is a different board?
In this case, select a variant that also has the FFat partition among the choices, or select a generic ESP32 dev-module so you have all the options available.

A curiosity... but why are you so sticky to the FFat partition? If you left LittleFS, the sketch would work as it was without having to specify the partition scheme every time

[MyRaspberry]

oh sorry,
-1- my idea was to test a new ESP32-S3 N16R8
with Arduino, so installed Arduino IDE 2.2.1 to learn that too.

-2- my first impression was that i can not use that N16R8
so i dig deeper and that leaded to using FFAT
but there i got the feeling that there is a big problem about up and download?
so i started to look and try FSBrowser lib.

as i am working and data-collecting to N16R8
in that bigger evaluation project already
status

-3- as described above, i did a separate test setup for your lib.
-a- laptop
-b- Arduino IDE 1.8.19
-c- a NODEMCU ESP32-32s v1.1
as i got the impression you need that Arduino /data/ dir thing
where i read that not work in the new IDE?

ok, i changed to LITTLE FS
and the old ESP32 is ok,

• login and change to my SSID
• upload all your files from /data
• found a new CSV file from now and growing
• CSV download ok
• change to static IP
  *had to do it 2 times, first i used 192.168.1.213 | 192.168.1.1 | NO ENTRY
  5:08:48.355 -> [I][esp-fs-webserver.cpp:143] startWiFi(): Manual config WiFi connection with IP: 192.168.1.213
  15:08:48.543 ->
  15:08:48.543 -> [I][esp-fs-webserver.cpp:173] startWiFi(): Connecting to 'kll-wlan_2.4G'E (794) esp_netif_handlers: invalid static ip
  15:08:49.550 -> ................
  15:09:04.573 -> [I][esp-fs-webserver.cpp:120] startAP(): AP mode.
  15:09:04.573 -> Server IP address: 192.168.4.1

second try used 192.168.1.213 | 192.168.1.1 | 192.168.1.1
again after board reset COM show that info,
BUT
http://192.168.1.213/
works from 2 computer.

ok, thank you so much

[cotestatnt]

as i got the impression you need that Arduino /data/ dir thing where i read that not work in the new IDE?

The FS upload plugin for IDE 1.8.x (which don't work with IDE 2.x) is no more necessary because you can upload your files through the esp-fs-webserver library's included web interface.

second try used 192.168.1.213 | 192.168.1.1 | 192.168.1.1

I don't think the subnet mask you used are a valid value. Usually I set as 255.255.255.0

If you edit manual IP or other parameters for no-DHCP configuration, you have to save configuration before start the new connection, otherwise on next restart it will not be used.

[MyRaspberry]

255.255.255.0
my bad

[MyRaspberry]

after some looks i got to say:

-a-

this project needs a FAVICON.ICO
like
with change in :
/data/index.htm
< link rel="icon" type="image/x-icon" href="assets/favicon.ico" >
and here my idea, pls you use mine if you like it, even i am not a real artist.
pls copy from :
here

-b-

but just found YOUR /edit page !! very good

i miss info about the 'LittleFS' DRIVE in PSFLASH

• space? total free used

-c-

this project csvLogger : funtion is impressive,
but the code is 3 level over my head to understand
if you look at my GIT you see i just managed to learn .h .cpp class
and have html download button browse select one file, but the way seems to be same?

server ON /edit [POST] >> handleFileUpload() >> server.upload()

all just hidden in your complex structure.
but i not see where you scan the PC selected /data/ dir and upload ALL files?
is it a ONE by ONE via a JS loop?

-d-

just setup other PC

• use Arduino IDE 2.2.1
• but still NODEMCU board
• project from your lib ( new installed )
• change to LittleFS
• erase all flash DISABLED

lost all old content, pls verify

looks like the new IDE also messes up about the data dir from example when save to own project

-e-

anyway

• we both got the poor FSBrowser to a FULLER USE
• and i am happy that using N16R8 via Arduino IDE is tricky but possible.

• • took me a week to find that out, funny that google could not help me with that, possibly my bad english.

thanks and greeting to Rome from Chiang Mai

[cotestatnt]

I'll try to respond to your comments

• A
  favicon.ico is something linked to the final application chosen by the user, so inserting it at library level seems useless to me. Of course I could put a default one to customize if necessary... I'll think about it.
• B
  the methods for retrieving this information is tied to the choosen filesystem. In the twin library async-esp-fs-webserver, is possible to assign a callback function for this task. I will include this possibility in this library soon
• C
  yes, it's all done with a Javascript function and with HTML input file element with id=listing
• D
  ok, I'll do some checking
• E
  Thank you from Rome for the valuable feedback

[MyRaspberry]

now i try

• Arduino IDE 2.2.1 ( after NODEMCU ESP32 S OK )

#include <FS.h>

#include <LittleFS.h>
#define FILESYSTEM LittleFS

• board ESP32-S3 N16R8
• erase all FLASH ENABLED

?? WHAT NOW??

• FLASH size 4MB
• Partition 4MB 1.5MB SPIFFS

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4

[I][esp-fs-webserver.cpp:120] startAP():	AP mode.
 Server IP address: 192.168.4.1

ESP Web Server started on IP Address: 192.168.4.1
Open /setup page to configure optional parameters
Open /edit page to view and edit files

also try:

#include "FFat.h"
#define FILESYSTEM FFat

FLASH size 16MB
Partiton 16MB 3 APP 10 FATFS


[E][esp-fs-webserver.cpp:24] begin():	Failed to open /setup directory. Create new folderESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x4201b1d4
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4

[I][esp-fs-webserver.cpp:120] startAP():	AP mode.
 Server IP address: 192.168.4.1

ESP Web Server started on IP Address: 192.168.4.1
Open /setup page to configure optional parameters
Open /edit page to view and edit files

• mobile wifi >> ESP_AP

• •            password 123456789
    

• • Authenticating..
• • Couldn't connect Try again

-1- pls help with the tools board settings for ESP32-S3
-2- why now login problems?

[cotestatnt]

I'm sorry something must have gone wrong with the latest commits...
for now I would suggest starting the connection manually with the classic WiFi.begin() and avoiding calling the library startWifi() method

I'll fix this bug and then I'll make a new release.

[MyRaspberry]

ok, n.p. can wait,
just wanted to confirm if the rest worked and run pure FFat tools over it

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4


================================
Chip Model: ESP32-S3
Chip Revision: 0
with 2 core
Flash Chip Size : 16777216 
Flash Chip Speed : 80000000 
Total Heap : 397212 
Free Heap  : 372320 
PSRAM Total :  8386279, Free : 8386019, Used : 260

Features included:
 
 2.4GHz WiFi
 Bluetooth LE
 
 

FFAT Total :   10240000, Free :   10231808, Used :       8192
Listing directory: /
  DIR : setup
Listing directory: /setup
  FILE: config.json	SIZE: 17

---

no offense, but that guy is wow

[MyRaspberry]

update:
try v 1.3.2

[cotestatnt]

Hi @MyRaspberry, thanks for your review!
I just published a new major release (due to the big and deep changes).

If you want test also this release, please keep in mind that some small changes are needed on on pre-existing sketches.
In any case, the advice is always to start with the examples included.

[MyRaspberry]

try v 2.0.0

[cotestatnt]

Hi @MyRaspberry
If you like, try the latest release.
I've fixed some of the observations you made such as wrong printing files order.
As regards the "Google" IP in access mode, it seems that with this trick you can make the captive portal work even on the most finicky devices (for example some Samsungs smartphones)

Furthermore, in the csvLogger.ino, I have included an example of how it is possible to insert a custom favicon encoded in base64.

The /edit web page must be explicitly enabled. However I have adjusted all the examples to include it (and possibly comment the related instruction)

[MyRaspberry]

v 2.0.1 : test 9 and 10