Clarifications and further information required in docs

[christopherthomson]

Feature Request

Feature Description

This looks like an excellent project and I'd love to get it running but am unclear about what configuration steps are essential for a minimum running app using the local non-docker option. I have developed some simple Flask apps but the configurability here is a bit beyond my experience. Could you please assist?

I have created a database in mysql, added user and grant privileges, then added db conn string to settings.py. I also added a pay period start date and a reconcile start date.

I installed all requirements using pip. I used
export FLASK_APP="./biweeklybudget.flaskapp.app"
flask run

I get the error:
Usage: flask run [OPTIONS]

Error: The file/path provided (./biweeklybudget.flaskapp.app) does not appear to exist. Please verify the path is correct. If app is not on PYTHONPATH, ensure the extension is .py

Adding the .py extension to the path still gives the same error. Any advice? Perhaps a bit of documentation for beginners :) Thanks.

[jantman]

@christopherthomson First off, great to hear from you! I created this app just for myself and you're only the second person who's given me indication that they may be running it.

Secondly, just a heads-up (this should be documented) the master branch in git is not always strictly usable. If you're going to give it a try, I'd highly recommend installing either from the PyPI release or running off of a release tag in the git repo.

As to the exact issue, how did you install biweeklybudget?

When I run locally for testing, I pretty much exactly follow the documentation, specifically:

1. Installation
2. Configuration (which it sounds like you've already done)
3. Setup / exporting the settings module
4. Running Flask

I believe the issue is that you're providing a file path to FLASK_APP but it wants a Python import path. As shown in the docs on running Flask, that should be export FLASK_APP="biweeklybudget.flaskapp.app".

[christopherthomson]

Thanks for your quick reply! I see I pasted in a variant I tried with './' in front. However, I've tried again and get the following:

(env) chris@zzz:~/Projects/biweeklybudget$ echo $FLASK_APP
biweeklybudget.flaskapp.app
(env) chris@zzz:~/Projects/biweeklybudget$ flask run
Usage: flask run [OPTIONS]

Error: The file/path provided (biweeklybudget.flaskapp.app) does not appear to exist. Please verify the path is correct. If app is not on PYTHONPATH, ensure the extension is .py

I installed using git clone, then checked out the 0.7.1 tag to a new local branch. I created a database in MySQL, but haven't created any tables manually. I used pip to install the packages in requirements.txt.

I haven't created a settings module as you suggest in the documentation, because I'm not sure where to start with this. I have just edited the settings.py file with my database connection details and exported the FLASK_APP environment variable using: export FLASK_APP="biweeklybudget.flaskapp.app" (though this doesn't seem to be found when I try flask run - I get the error above).

I am trying to run the app from the top level 'biweeklybudget' directory.

Any further suggestions gratefully received! I can also try the Docker route, but am hoping to get started as simply as possible.

[jantman]

@christopherthomson I believe the missing step in the above is that after using pip to install requirements.txt, you need to run python setup.py develop to "install" biweeklybudget itself. I believe that's the cause of the error about the app not being on PYTHONPATH.

You'll need some settings beyond the database connection string. I'd highly recommend against editing settings.py directly. In terms of simplicity, when you're installing from git locally, there are two options:

1. Copy settings_example.py to some other name like mysettings.py, edit that file, and then export SETTINGS_MODULE=biweeklybudget.mysettings
2. Don't worry about editing settings files at all, export SETTINGS_MODULE=biweeklybudget.settings_example, and then pass all of the configuration as environment variables. If you do this, I'd recommend writing a bash wrapper script that sources the virtualenv, cd to the right directory, exports your settings environment variables, and then runs flask.

For what it's worth, if you're familiar with Docker, that's definitely the easier method.

[christopherthomson]

Okay, thanks so much for your further advice. I hadn't run setup.py so that probably explains my difficulty. But, I got stuck getting the database connection details right, so tried the Docker route instead. I've eventually got that up and running and learned lots about Docker in the meantime!

I've got an instance running, and am curious how to use the OfxBackfiller functionality. I copied an ofx file ('backfill.ofx') to the container and tried /app/bin # ./ofxbackfiller -s /backfill.ofx but nothing happened. However the loaddata program loads sample data well (though I'm not sure how to delete it again!). I'll play around with it a bit further and see. If there's anything you can suggest on this, let me know! Cheers.

[jantman]

@christopherthomson I apologize that the docs aren't as good as they could be... Aside from one person who opened an issue asking for internationalization support, you're the only person who's contacted me about trying to run this app... I developed it for my personal use, and didn't really know if anyone else would be interested (and it's pretty rough around the edges).

loaddata is intended for development purposes, and loads the sample data that's used for most of the automated tests and the screenshots. Once the data is in your database, you'd either need to drop and then recreate the database, or drop all of the tables (the latter is a bit of a pain because of the foreign key relationships).

Ofx in Docker is... a bit of a pain. The assumption is that your statements will be stored somewhere on your host filesystem and mounted into the Docker container. An example would be:

1. Create a directory to keep your statements in on your host OS, e.g. /home/chris/statements.
2. Mount that directory into the docker container at some arbitrary path. You'd run the container with an additional argument, e.g. -v /home/chris/statements:/app/statements (that mounts /home/chris/statements on the host OS at /app/statements in the Docker container).
3. Update your settings file or environment variables and set STATEMENTS_SAVE_PATH = '/app/statements' (i.e. set STATEMENTS_SAVE_PATH to the path inside the Docker container).

So that's the first half - configuring and setting up the system. The second half is actually making it work for accounts. To do that:

1. In order for it to pick up OFX files for an account, the account needs to have something (a non-empty JSON object) in the ofxgetter_config_json field. If you're not using ofxgetter, you can just set this to something like {"foo": "bar"}
2. The backfill script looks for OFX files on the filesystem in paths matching STATEMENTS_SAVE_PATH/ACCOUNT_NAME/*.ofx. The actual logic of ofxgetter is to find every Account in the database that's marked for OFX (ofxgetter_config_json not empty or null) and loop through them one at a time, trying to find a subdirectory of STATEMENTS_SAVE_PATH with the same name as the account. If that's found, it examines every .ofx or .qfx file and attempts to backfill them.

You don't have to mount the statements save path in from the host, you can copy them to the container if you want. But I think the problems you're having - that aren't documented yet - are because (1) a filesystem hierarchy of SAVE_PATH/AccountName/*.ofx is expected, (2) the -s flag to ofxgetter and ofxbackfiller specifies that top-level SAVE_PATH directory not a single file, and (3) it doesn't look at every file, it looks at subdirectories that match up to account names.

Also note that file path on disk is used as a key in the database - if you put an OFX file at SAVE_PATH/AccountName/foo.ofx, it will never load another OFX file from that path.

Apologies for this being so rough around the edges, but I'll definitely go through this issue and try to document all of these confusing points.

[christopherthomson]

Don't apologise, it's all good. It's a pretty complex app and impressive you've built so many tools around it already. I've got a busy couple of days ahead but will take a closer look at your advice later in the week. The ofxbackfiller looks really handy. Thanks again.

[jantman]

Sure, I'm glad it looks useful to you! I know we're pretty far apart in time zones, but I'll do my best to respond if/when you have a chance to give it a try.

[zalcock]

Good day Jantman.

I would like to know don't you have detail installation steps to get the biweeklybudget up. Reason why i'm asking this is because i'm new to this and would really like to test your program

[jantman]

@zalcock There's documentation available at http://biweeklybudget.readthedocs.io/en/latest/

Beyond that, no, there's nothing more detailed. This project is quite unofficial and really only intended for people familiar with Python, Flask, etc.

[jantman]

@christopherthomson @zalcock have each of you either gotten this working, or decided not to? Is there anything left to do for this issue, aside from maybe updating the documentation to include some of the discussion here?

[jantman]

FYI I renamed this, and will leave it open until I update the documentation with at least the information in this issue so far.