Would people who use this want our full-blown Django admin feature set?

[palewire]

I have some code I havn't merged in that would replace the text-editor and command-line style here with a Django admin that replaces the YAML and bakes your files up to S3 when you click save. It requires more setup (You need a Celery job server yet) and introduces a lot of complexity, but does it make it easier to get non-programmers using this thing.

[palewire]

Please leave comments here and tell me what's up.

[particleandparcel]

I think I would, assuming my Python/Django is up to the task.

There's a lot of stuff involved in using the standalone TableStacker that is over non-programmers' heads, like YAML or running the script to bake and publish a table. It keeps them from accessing tools like this. The broader question is whether Table Stacker is, or could be, used by non-technical users. But right now, I doubt it is.

[palewire]

One downside to the admin though is that you still have to write a little YAML to set the "column_options."

That's because there is no way I know of in Django to set dynamic options, in this case based on the CSV uploaded by a user. The headline and information about the "table" as a whole are simple, but how you set options on a column-by-column basis (i.e. "What order should the columns go in?") aren't handled by any default Django options I know of.

[tommeagher]

Sorry for the delay in responding. Been a little tied up with a newborn who just recently came home.

I'm comfortable using TableStacker with the CLI, but one of my pipe dreams is to have it more accessible for everyone in the newsroom, particularly the folks on our continuous news desk who handle most of the day-to-day web production. For simple tables, I'd like for those folks to be able to publish themselves. How involved is the YAML in the admin setup? If it's along the lines of the formatting I'm already doing, it doesn't seem like an insurmountable hurdle.

I've never worked with Celery, but I'm willing to learn if it would democratize this work a bit. Panel at NICAR?

[palewire]

Basically, the extra setup is that you'd have to install Rabbit MQ's job server (sudo apt-get install rabbitmq-server) and Celery (pip install django-celery), which is a Django wrapper for it. What it does is create a job when you hit save in the admin that will bake out the changed pages and sync them up with S3 without the user knowing what's going on.

[tommeagher]

You make it sound so simple. What's the admin YAML like?

[palewire]

Basically, you still have to learn enough to do the column_options part in existing YAMLs. So there's like a big text field in the admin where you have to do those bits. Everything else is handled with standard Django admin widgets. This is the part I mean:

    columns:
      - Collective
      - Address
      - Applied for lottery
      - Ordered to close
    style:
      Collective: 'text-align:left; width:450px;'
      Address: 'text-align:left;'
      Applied for lottery: "text-align:center; width:150px;"
      Ordered to close: "text-align:center; width:150px;"
    sorted_by:
      - Collective: ascending
    formatting:
      Applied for lottery:
        method: bubble
      Ordered to close:
        method: checkbox
    per_page: 500

[tommeagher]

Not ideal, but for folks comfortable in html, it's certainly doable with a wee bit of training.

[palewire]

Yeah, it sucks. It's because there is no way I know of in Django to set dynamic options, in this case based on the CSV uploaded by a user. How you set options on a column-by-column basis (i.e. "What order should the columns go in?") is still an open challenge to me. If you have ideas how that could be admin-ified, let me know.

[tommeagher]

I have no idea. I'm a Django neophyte, but I'll give it some thought. If I have any inspiration, I'll send them your way.

[palewire]

In my pea-brain, it seems like you'd have to read the uploaded CSV and then create a configuration interface on the fly depending on the headers of the CSV. Or have a two-step upload-and-configure process.

[kevinschaul]

Jumping in here, but we've used (and loved) table-stacker at MinnPost, but it's always reporters sending me the csv, and I do the rest. I successfully taught one of our reporters the basic parts of the YAML file, and I agree that it isn't insurmountable for reporters/editors to learn this. A slick interface would be welcomed, though. Doesn't seem impossible. Worth it? Maybe.

Either way, the Django admin interface would be great.

[tommeagher]

I think a two-step process would work. Users will want to be able to customize the formatting for each column themselves, even if they don't want to write the code to do it.

Is it possible to have an admin user upload a CSV, and the app would use CSVkit or another Python CSV module to read the column heads?

Then, could you dump those column heads into a custom admin view and template that would let you choose the style, formatting and sorting from drop-down menus?

[palewire]

Maybe that's the trick, some custom admin widget that like a wizard runs you through a series of popups or something, but which is secretly editing the YAML config underneath.

[palewire]

Check that. Not like a wizard. Like a boss. Much better.

[tommeagher]

Boss. I like that.
Could you do this?

On Jan 17, 2012, at 9:36 PM, Ben Welshreply@reply.github.com wrote:

Check that. Not like a wizard. Like a boss. Much better.

---

Reply to this email directly or view it on GitHub:
#88 (comment)

[palewire]

It's a maybe. I can port our existing thing, with the in-admin YAML editing, and then we can try to figure it out. It sounds like something that's doable, but I've never done anything like it before. I think you'd need to write a goodly amount of JavaScript for a custom admin widget that would interact with the CSV file. Though how exactly you'd open it up I don't know yet. Because I don't think the CSV is posted to the database and available to the Python code until after you save the admin record for the first time.

[palewire]

You could try to separate the CSV upload from the table's admin, maybe. Like make a CSV model where those are stored, and then the Table model has a Foreign Key linking to that. So that you'd have to fill out an inline or separately upload a CSV in another admin window before you could configure the public-facing table. Now, my system does all that in one model and with one admin.

[palewire]

Another angle is you cut the CSV upload, and instead make a new magic widget that's a TextField. When CSV data is pasted in, it magically susses out all the columns and such, which could then be fed into the configuration wizard boss. That's kind of what BatchGeo.com does.

[tommeagher]

That's a great idea. Reminds me in a way of MrDataConverter
https://github.com/shancarter/Mr-Data-Converter

[palewire]

Advice on making that kind of magic JS box work?

Basically, we'd need that but wired to a Django model field through the admin's widget framework. And then some JS in the change form would pop up a form that will fill out the column options, which would them have to be bound to the YAMLField containing that config. That second part could get pretty complicated too. It would have some complex forms to let you choose and order the columns, set CSS styles for selected columns somehow, pick and configure formatters, set a sort order, etc.

[particleandparcel]

Wow. I missed out on a lot of this debate! I think this is definitely the
way to go. Think of how Access and Outlook handle CSV/TSV imports: first it
figures out what columns are which, and then you tell it what type of data
it is and which formatter (if applicable) you want. This would also give
you the flexibility to allow custom delimiters on import, i.e., if I have a
tab-delimited file, I wouldn't need to teach someone how to import one into
Excel and export a CSV.

I wrote a bit of code to do some of what we're describing a few months
back, for a presentation for a student journalists conference. But it's all
in PHP, so it's probably not real useful to offer. Maybe the JavaScript
bits like the CSV parsing and custom delimiter code.

[mikejcorey]

I think I'll chime in for a 2-step process being a good way to go. And using a Web interface (Django admin or otherwise) is probably the only way to get non-developers on board. I wish it weren't but command line is probably always going to be a bridge too far for a certain segment. I was a Web designer for years without knowing any command line, though I shouldn't have been able to get away with it.