/
presentation.html
1188 lines (871 loc) · 41.4 KB
/
presentation.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<meta name="keywords" content="remark,remarkjs,markdown,slideshow,presentation" />
<meta name="description" content="A simple, in-browser, markdown-driven slideshow tool." />
<title>Presentation</title>
<link href='https://fonts.googleapis.com/css?family=Inconsolata' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="/assets/css/presentation_style.css"></link>
<style type="text/css">
.remark-code, .remark-inline-code {
font-family: "Inconsolata", "Courier";
}
.remark-slide-content > ul li ul li {
font-size: 18px;
}
.ex {
background-color: #E3E7F0;
color: black;
padding-left: 5px;
}
.left-column {
width: 50%;
}
.right-column {
width: 50%;
}
#slide-roadmap .left-column,
#slide-roadmap .right-column {
margin-top: -3rem;
}
#slide-databinding #circle {
width: 40px;
height: 30px;
top: 66%;
right: 48%;
opacity: 0.3;
position: absolute;
border: 10px solid red;
-moz-border-radius: 50px;
-webkit-border-radius: 50px;
border-radius: 50px;
}
.img100 img {
width: 100%;
}
.bold {
background-color: antiquewhite;
}
.table table {
width: 100%;
font-size: 0.9rem;
}
.footnote {
position: absolute;
color: #A8A358;
right: 1.5em;
top: 1.5em;
}
</style>
</head>
<body>
<textarea id="source">
layout: true
class: left
background-image: url(../assets/images/logos.png)
---
name: intro
class: center
<br/>
# Data Visualization on the Web with D3 and Leaflet
<br/>
#### Shruti Mukhtyar
Web Developer
Geospatial Innovation Facility, UC Berkeley
Projects: [Cal-Adapt](cal-adapt.org), [VTM](http://vtm.berkeley.edu/)
<br>
Twitter: .bold[_@mapchitra_] .bold[_@cal_adapt_]
---
<br/>
## Today's Plan
- Brief overview of Data Visualization
- Where does D3 fit in?
- Whirlwind introduction to Web Development
- Explore D3 Core Concepts and build a bar chart
- Web mapping with Leaflet
- Discussion (Visualization, Web Development, Learning, Deeper dive into some examples, etc.)
---
class: center
<br/>
# Brief Overview of Data Visualization
<br/>
---
## What is Data Visualization?
- Graphical display of abstract information for communication and sense-making
## Goals
- Using graphics with or without explanatory texts for .bold[Communication]
- Displaying data for exploratory .bold[Analysis]
- [The Changing Goals of Data Visualization](https://eagereyes.org/criticism/changing-goals-data-visualization)
???
main goal of data visualization is to communicate information clearly and effectively through graphical means
---
## Types of Quantitative Messages (Stephen Few)
- .bold[Timeseries]: A single variable is captured over a period of time
- .bold[Ranking]: Categorical subdivisions are ranked in ascending or descending order
- .bold[Part-to-whole]: Categorical subdivisions are measured as a ratio to the whol
- .bold[Deviation]: Categorical subdivisions are compared against a reference
- .bold[Frequency Distribution]: Number of observations of a particular variable for given interval
- .bold[Correlation]: Comparison between observations represented by two variables
- .bold[Nominal Comparison]: Comparing categorical subdivisions in no particular order
- .bold[Geographic]: Comparison of a variable across a map or layout
???
Author Stephen Few described eight types of quantitative messages that users may attempt to understand or communicate from a set of data and the associated graphs used to help communicate the message:
---
class: center
## Form follows function
.footnote[[Link](http://www.vizualism.nl/infographic-taxonomy/)]
???
how the visualization method is defined by a combination of the basic questions and the structure of the dataset.
---
## Today's goal
- What kind of visulizaion to build? <span style="font-size:1.5em;">✗</span>
- [List of books](https://www.visualcinnamon.com/resources/learning-data-visualization/books)
- Alberto Cairo, Cole Nussbaumer Knaflic, Andy Kirk
- Edward Tufte, Stephen Few, Tamara Munzner,
- How do you go about building the visualization you want? <span style="font-size:1.5em;">✔</span>
---
## Why publish on the web?
- Fastest way to reach a larger audience
- Working with web-standard technologies means that your work can be seen and experienced by anyone using a recent web browser, regardless of the operating system and device type.
- Avoiding proprietary software (e.g. GIS)
---
## Technology Strategy
- Who's your target audience?
- How much data to you have? Where is it?
- Is the web map/dataviz going to part of an existing website?
- Do you need to support tablets and mobiles?
- Do you need to support offline access?
- Public or private?
## Data Strategy
- Data has a longer life-cycle
- Web presentations of data age quickly
???
One of the most important skills you'll learn making websites is this: Don't push for a technology because it's popular or considered cool. Or because it's familiar to you. Find the right solution to the actual problem at hand. Consider the long term plan.
Who will take care of this site for the next 10 years? How will technology choices make that harder or easier? How can you make sure limited time & resources are spent on what's most important (like making content, perhaps) and not on what's not (like endless security updates)?
How can you keep things as simple as possible, while having the power and complexity that is needed?
These are hard things to learn. Much more than a new language. Experience will teach you.
---
class: center, img100
## Visualization Tools
.footnote[[Link](http://selection.datavisualization.ch/)]
---
class: center, img100
## Visualization Tools
.footnote[[Link](http://selection.datavisualization.ch/)]
---
class: center, img100
## Visualization Tools
.footnote[[Link](http://selection.datavisualization.ch/)]
---
class: center, img100
## Visualization Tools
.footnote[[Link](http://selection.datavisualization.ch/)]
---
class: center, img100
## Visualization Tools
.footnote[[Link](http://selection.datavisualization.ch/)]
---
class: center
<br/>
# Where does D3 fit in?
<br/>
---
## What is D3?
- From .bold[d3js.org]:
- D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.
## Why learn D3?
- Create highly customizable and interactive visualizations on the web
- Knowing D3 = Hirable skills
- Opportunity to learn web development
- Thousands of examples and a great open source community
- [5 reasons to learn D3](http://d-miller.github.io/Why-Learn-D3/)
???
Use JavaScript and HTML and libraries such as d3.js to create highly customizable visualizations.
Learning to make even a simple chart can take time, but you develop core web and coding skills along the way.
When learning D3, you learn many other web skills involving JavaScript (known as the “programming language of the web”) and HTML. Even if you decide you hate D3, knowing those skills will be useful in any other project involving web development.
You learn these broader skills because D3 exposes you to the internals of a webpage. In more technical terms, D3 doesn’t use an “intermediate” representation. It instead directly manipulates elements on the Document Object Model (DOM), which is basically just a fancy term for everything that is currently shown on a webpage.
---
## Linked Beeswarm Plot
.footnote[[Link](http://bl.ocks.org/HarryStevens/bd3ce1862ccfbb5b0c78b56aca8fca6c)]
???
The beeswarm plot is a one-dimensional scatter plot like "stripchart", but with closely-packed, non-overlapping points.
Essentially beeswarm plot is used to visualize distributions similar to stripchart, histogram or box and whisker plot. The difference between beeswarm plot and other traditional chart type that visualize distributions is that beeswarm plots the data on the single axis and then offsets in the other direction to show volume or counts.
---
class: img100
## Face of Fracking
.footnote[[Link](http://bl.ocks.org/HarryStevens/bd3ce1862ccfbb5b0c78b56aca8fca6c)]
---
class: img100
## American Panorama - Foreign Born Population
.footnote[[Link](http://dsl.richmond.edu/panorama/)]
???
American Panorama is an historical atlas of the United States for the twenty-first century. It combines cutting-edge research with innovative interactive mapping techniques.
See how the foreign-born density of counties in the US changes over time. Green circles indicate how many people in the US were born in that country for a given year.
---
## Simpson's Paradox
.footnote[[Link](http://vudlab.com/simpsons/)]
---
## Generative Art with D3
.footnote[[Link](https://learningd3.com/blog/generative-art/)]
???
Definition of generative. : having the power or function of generating, originating, producing, or reproducing.
---
## Optionally...
- Software
- Tableau, Excel, GIS software
- Data visualization servies
- [Datawrapper](https://www.datawrapper.de/), [RAW Graphs](http://rawgraphs.io/)
- R or Python
- Associated visualization packages such as ggvis, rCharts, and Shiny
- Create interactive graphics but less customization options and/or limited pre-existing graphics types (e.g., bar charts).
---
class: img100
## Visualizing Friendships - R
.footnote[[Link](https://www.facebook.com/notes/facebook-engineering/visualizing-friendships/469716398919)]
???
Createdin R, locality of friendship
---
class: img100
## Five Years of Drought - ArcGIS Pro
.footnote[[Link](https://adventuresinmapping.com/2016/07/12/five-years-of-drought/)]
???
This is a map showing over five years of drought data (285 weeks, combined into a single view) in the United States.
The dots are proportionally sized by the amount of time over the past five years spent in drought (the largest dots representing 80% – 100% of the time), and the color is a weighted value of the overall intensity of those droughts (deep purple indicates frequent “exceptional” drought, the most severe category).
---
class: center
<br/>
# Whirlwind Introduction to Web Development
<br/>
---
class: img100
name: roadmap
## Roadmap for Learning Web Development
.footnote[[Link](https://github.com/kamranahmedse/developer-roadmap)]
.left-column[
- Frontend
- HTML5, CSS3, JavaScript
- Frontend frameworks
- SVG, Canvas, WebGL
]
.right-column[
- Backend
- Python, Java
- Backend frameworks
- Web Servers
]
---
class: center
.footnote[[Link](https://github.com/kamranahmedse/developer-roadmap)]
---
## Quick Start Technology Stack
- Data, Middleware & User Interface
- ArcGIS Online
- CARTO
- MapBox
- Google
- Tableau, RStudio, [Datawrapper](https://www.datawrapper.de/), [RAW Graphs](http://rawgraphs.io/)
- Embed widgets into your own website
---
## Hybrid Technology Stack
- Don’t want to maintain database/mapping servers?
- Use data from Web API's
- Use web services from ArcGIS Online, CARTO, MapBox, Google, etc.
- Build custom user interface with JavaScript libraries
- Don’t want to write Javascript/HTML/CSS?
- Build your web app with Rshiny; host your data and app on a server running Shiny Server
- Build your web app with Python packages (Folium, Bokeh), deploy on a web server
- Interactive maps and data with Google Sheets
---
## Custom Technology Stack
---
## Quick Review
- Client & Server
- Web Developer Tools
- HTML
- CSS
- DOM
- JavaScript
- Web Graphic Formats - SVG and Canvas
---
## Client & Server
- HTTP, language of the web
- Browser sends HTTP GET Request. Server sends response for each request with content and/or status message.
- [HTTP Basics](https://www.ntu.edu.sg/home/ehchua/programming/webprogramming/HTTP_Basics.html)
???
When a link is clicked, the browser creates an .em[HTTP Get Request] that is sent to the server. Server responds. Browser displays the page. Sees that it needs a few more things - e.g. images, fonts, other files. Browser sends new a GET request for each item. Server sends response for each request with status code (200 - found, 304 - retrieving from browser cache, 404 - not found, 500 - server error )
---
## Web Developer Tools
- Code Editor
- Sublime Text, Atom, PyCharm, Vim, Notepad++, IDLE, Gedit, TextMate
- Syntax highlighting, indentation, autocomplete, bracket matching
- Run interpreters, debuggers
- Browser
- Developer Tools - press `Ctrl+Shift+I` (or `F12`) in Chrome
- More info on Chrome Developer tools [here](https://developers.google.com/web/tools/chrome-devtools/)
- [Github](https://guides.github.com/activities/hello-world/)
- Code hosting platform for version control and collaboration
---
### Hyper Text Markup Language (HTML)
- HTML is not a programming language; it is a markup language
- .bold[Role » Describes content]
- HTML Elements consist of a opening tag, the content, and a closing tag
```xml
<ul>
<li><a href="gif.berkeley.edu">Home Page</a></li>
<li><a href="gif.berkeley.edu/people">Staff</a></li>
</ul>
<div id="map" class="center" style="height:500px;"></div>
```
- HTML5 Resources
- [Dive into HTML5](http://diveintohtml5.info/)
- [Mozilla Developer Network](https://developer.mozilla.org/en-US/docs/Learn/HTML)
???
HTML and its DOM model, of course, provide the foundation on which any web based project must run. D3 provides terse APIs for interacting with the DOM, and some D3 projects are built entirely with native HTML elements. That said, most D3 visualizations create SVG output, which we'll discuss briefly below.
---
## Cascading Style Sheet (CSS)
- Not a programming or markup language
- .bold[Role » Describes presentation of a document written in HTML or XML (e.g. SVG)]
- Selectors, properties, and values
- Properties and Attributes
```xml
#map {
height: 500px;
}
h1 {
font-family: "Helvetica", "sans-serif";
font-weight: bold;
background-color: #000;
}
.chart {
float: left;
}
```
- CSS3 Resources
- [Mozilla Developer Network](https://developer.mozilla.org/en-US/docs/Learn/CSS)
???
Whether your project targets the DOM or SVG, you will use CSS to style it. D3 provides APIs for adding and removing CSS classes as well as manipulating individual style properties, so as long as you know the basics you shouldn't have any trouble.
---
name: dom
## Document Object Model (DOM)
- .bold[Structured representation of the document (a tree) created by the browser]
- HTML you write is parsed by the browser and turned into the DOM
- Programming interface for HTML and SVG documents
- JavaScript manipulates the DOM
???
When a browser displays a document, it must combine the document's content with its style information. It processes the document in two stages:
The browser converts HTML and CSS into the DOM (Document Object Model). The DOM represents the document in the computer's memory. It combines the document's content with its style.
The browser displays the contents of the DOM.
Understanding the DOM helps you design, debug and maintain your CSS because the DOM is where your CSS and the document's content meet up.
---
## Conceptual Web Page
```xml
<!DOCTYPE html>
<html>
<head>
<!-- head content -->
<style type="text/css">
div {
color: red;
}
</style>
</head>
<body>
<h1>Header text here</h1>
<div>
<h2>Subheader text here</h2>
<p>Paragraph text here</p>
</div>
<div>
<p>Another paragraph here</p>
</div>
<script>
<!-- JavaScript content -->
</script>
</body>
</html>
```
---
## The DOM for conceptual web page
- [Understading the DOM](https://www.digitalocean.com/community/tutorial_series/understanding-the-dom-document-object-model)
---
## JavaScript
- "Easy to learn, hard to master"
- .bold[Role » Creating interaction]
- Interpreted by your browser
- Asychronous (code executes in background after client-browser recieves data from server)
- Get familiar with:
- Working with json objects
- Array functions (forEach, filter, map)
- Method chaining, Callbacks, Closures, Modules
- JS Resources
- [Mozilla Developer Network](https://developer.mozilla.org/en-US/docs/Learn/JavaScript)
???
JavaScript is arguably more difficult to learn than related technologies such as HTML and CSS. Before attempting to learn JavaScript, you are strongly advised to get familiar with at least these two technologies first, and perhaps others as well.
---
## SVG
```xml
<svg width="300" height="180">
<rect x="10" y="20" width="20" height="50" fill="blue"
stroke="red" stroke-width="1"/>
<circle cx="100" cy="100" r="25" fill="red"
stroke="#ddd" stroke-width="5"></circle>
<g transform="translate(5, 15)">
<text x="0" y="0">My graphic</text>
</g>
<g transform="translate(5, 55)">
<!-- M: move to (jump)
L: line to
Q: curve to (quadratic) -->
<path d="M0,50 L50,0 Q100,0 100,50"
fill="none" stroke-width="3" stroke="black" />
</g>
<g transform="translate(5, 105)">
<!-- C: curve to (cubic)
Z: close shape -->
<path d="M0,100 C0,0 25,0 125,100 Z" fill="black" />
</g>
</svg>
```
- Play with this code on [SVG Graphic Primitives - JSFiddle](https://jsfiddle.net/mukhtyar/04m3xn4r/)
???
Where HTML has the <div> and <span> tags, SVG has the <g> tag for an arbitrary group. You’ll see <g> a lot in D3 examples. They’re great for applying styles to a group (including re-positioning the groups).
The <text> tag is good for simple labels. The <path> tag is powerful but complex, it can be used for either lines or arbitrary filled-in shapes depending on the styling.
SVG is a "language for describing two-dimensional vector and mixed vector/raster graphics in XML". Really rolls off the tongue, doesn't it?
All you really need to know is that vectors provide resolution independence, allowing you to create graphics that look good everywhere from phones to giant desktop monitors. Additionally, SVG graphics do not participate in the browser's standard layout flow.
What does that mean? Well, when you create an SVG element using markup like <svg width="300" height="300"></svg>, that 300x300 block will appear in the browser's layout flow just like any other element. If the tag was defined after a <div> tag, the SVG will be rendered below the div. Tags defined after the SVG tag will be rendered below it. Within that block, however, there is no automatic layout to prevent items from overlapping. If you create two circles with the markup <circle r="20" /><circle r="20" /> they will be rendered in the same exact spot, right on top of one another. While this may seem inconvenient at first, it actually provides an incredible amount of flexibility. Avoiding overlap requires explicit coordinates but SVG graphics are capable of extraordinary expressiveness through layering, opacity, interactivity, and fluid animation.
---
## SVG (contd.)
- Coordinate system, top-left is (0, 0)
- Depth depends on order in which shapes are drawn
---
## SVG v/s Canvas
- HTML5 Canvas is a raster based format for drawing on the web
- You can draw raster graphcis with D3
- You can only get pixel values on click event
- Faster
- SVG (Scalable Vector Graphics) is a vector based format for drawing on the web
- HTML has div and span, etc.; SVG has circle and rect, etc.
- SVG is a DOM for graphical elements
- You can attach events to SVG elements
- Most D3 examples work with SVG
---
## HTML, CSS and JavaScript playgrounds
- [Codepen](codepen.io), [JSFiddle](https://jsfiddle.net/), [JS Bin](https://jsbin.com/?html,js,output)
- Open this [Codepen link](https://codepen.io/mukhtyar/project/editor/XaboqA/) in a new tab to explore how HTML, CSS and JavaScript work together.
- In-browser code editor built for creating and sharing d3.js examples
- [Blockbuilder](http://blockbuilder.org/)
- Integrated with Github and bl.ocks.org
- Or set up a local testing server on your machine
???
At CodePen.io, they describe themselves as a “playground for the front-end side of the web.” CodePen is great for testing out bugs, collaborating, and finding new inspiration. It works by allowing you to create “pens”, which are sets of HTML, CSS, and JavaScript.
---
## Start a local testing server and review final project
- Set up a simple local HTTP server on Windows
- [How do you set up a local testing server?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/set_up_a_local_testing_server)
- Python library comes with a `SimpleHTTPServer` module
- For today we will use Python installed by ArcGIS on the lab computers
- Open Windows Command Prompt
- Change directory to the workshop folder
```javascript
cd Documents\workshop-intro-d3-leaflet
```
- Start the local server using python library installed by ArcGIS
```javascript
C:\Python27\ArcGIS10.4\python -m SimpleHTTPServer
```
- Open a browser. Type `localhost:8000` into the url bar at top. You should see a list of directories & files. Click on the `final` directory
---
## Web Development Resources
- If you are going to develop medium to large scale web apps learn more about using front-end development tools.
- [Getting Started Web Development Guide](http://jlord.us/blog/getting-started.html)
- [Front-end Handbook](https://www.gitbook.com/book/frontendmasters/front-end-handbook/details)
- Google Search
- Favor results from Stack Exchange, Mozilla Developer Network, CSS-Tricks
- [caniuse](caniuse.com) - Check which browsers support what features
- Web Design
- [A List Apart](http://alistapart.com/)
- [Webdesigner News](http://www.webdesignernews.com/) - curated stories
- [Eloquent Javascript](eloquentjavascript.net)
- [Egghead.io](https://egghead.io/) - Bite size web dev video training
- Lynda.com, Frontend Masters, CodeAcademy, Udacity, Coursera, etc.
---
class: center
<br>
# Explore D3 Core Concepts and build a bar chart
---
## D3 (Data Driven Documents)
- [D3.js](https://d3js.org/) is a javascript for producing dynamic, interactive data visualizations in web browsers.
- Built on the fundamental web standards of HTML, CSS, SVG, and JavaScript.
- Created by [Mike Bostock](https://bost.ocks.org/mike/), Vadim Ogievetsky and Jeffrey Heer (all at that time were part of the Stanford Visualization Group).
???
D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG, and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.
D3 is not a monolithic framework that seeks to provide every conceivable feature. Instead, D3 solves the crux of the problem: efficient manipulation of documents based on data. This avoids proprietary representation and affords extraordinary flexibility, exposing the full capabilities of web standards such as HTML, SVG, and CSS. With minimal overhead, D3 is extremely fast, supporting large datasets and dynamic behaviors for interaction and animation. D3’s functional style allows code reuse through a diverse collection of components and plugins.
Embedded within an HTML webpage, the JavaScript D3.js library uses pre-built JavaScript functions to select elements, create SVG objects, style them, or add transitions, dynamic effects or tooltips to them. These objects can also be widely styled using CSS. Large datasets can be easily bound to SVG objects using simple D3.js functions to generate rich text/graphic charts and diagrams. The data can be in various formats, most commonly JSON, comma-separated values (CSV) or geoJSON, but, if required, JavaScript functions can be written to read other data formats.
D3 is built on the fundamental web standards of HTML, CSS, SVG, and JavaScript. For the purposes of this course it is only assumed you have a basic understanding of HTML, CSS, and JavaScript. SVG will be taught from first principles as needed.
---
## What does it do?
- It is *not* a graphics library, it is *not* a charting library, it is a general purpose data visualization library
- Has everything you need for visualizing complex data BUT you will not find commands like barchart, scatterplot, or piechart, or even map
- Visualizes data using web standards (HTML, Javascript, CSS, SVG)
- Heavily used in data journalism and visualization. New York Times has been the pioneer in data visualization and data journalism.
- Most and foremost a DOM manipulation library (in this regard similar to jquery).
- Provides handy utilities for processing data (array, time series, geo data)
- Comes with a lot of functions and methods to create mapping functions that will map your data directly to properties on html elements
- Filled with algorithms (voronoi, quadtrees, circle fitting, convex hull, projections)
---
### Learning D3
.bold[One of the most interesting aspects about d3 is that it is the intersection between design and code, math and art, data and story - Ian Johnson]
- Very active user community in Bay Area.
- [Bay Area d3 User Group](http://www.meetup.com/Bay-Area-d3-User-Group/).
- Rich ecosystem of examples showing visualization types, coding techniques:
- [blockbuilder.org/](http://blockbuilder.org/)
- [bl.ocks.org/](http://bl.ocks.org/)
- [Navigating the D3 Ecosystem](http://enjalot.github.io/talks/navd3eco/)
- How do I learn D3.js? [Quora thread](https://www.quora.com/How-do-I-learn-D3-js)
- D3 version 3 not the same as .bold[D3 version 4]
???
Unfortunately, D3 also has a bit of a reputation as being hard to learn, but it doesn't have to be. D3 is built on a small handful of core concepts and once you understand them you can begin working toward mastery.
---
## Resources
- Books
- [Interactive Data Visualization for the Web](http://alignedleft.com/work/d3-book-2e)
- [D3 Tips and Tricks](https://leanpub.com/d3-t-and-t-v4)
- Curated lists of data viz and data viz research
- [visualisingdata.com](http://www.visualisingdata.com/)
- [The Chartmaker Directory](http://chartmaker.visualisingdata.com/)
- [Mike Bostock Blog](https://bost.ocks.org/mike/)
- For geographic visualizations
- [Learning D3.js 4 Mapping](https://www.amazon.com/Learning-D3-js-Mapping-cutting-edge-visualizations/dp/1787280179/ref=pd_lpo_sbs_14_t_2?_encoding=UTF8&psc=1&refRID=0555NNJZQ8NJH0Q0R5SD)
- [Jason Davies](https://www.jasondavies.com/maps/), [Jason Davies Code Blocks](https://bl.ocks.org/jasondavies)
- [API documentation](https://github.com/d3/d3/blob/master/API.md)
---
## Core Concepts - Declarative API
- .bold[Declarative]: Tell D3 what you want, not how you want it done
- .bold[API]: Methods and objects you use to interact with software
- e.g. keyboard interface for a computer
- Describe a relationship between data and UI (User Interface) elements, D3 handles implementation details
```javascript
d3.selectAll('p').style('color', 'red');
```
- Exercise: [D3- Declarative API](http://blockbuilder.org/mukhtyar/89e0407a1828d59c72e7)
---
class: table
## Core Concepts - Selections
- D3 provides shorthand for selecting and manipulating DOM objects (similar to jQuery)
- .bold[d3.select()], .bold[d3.selectAll()]
- Modify properties of selections
- Create/Read/Update/Delete (CRUD)
- Using Getters and setters
|Name|Behaviour|Example|
|--- |--- |--- |
|.style|Update the style|d3.selectAll('circle').style('fill', 'red')|
|.attr|Update an attribute|d3.selectAll('rect').attr('width', 10)|
|.classed|Add/remove a class attribute|d3.select('.item').classed('selected', true)|
|.property|Update an element's property|d3.selectAll('.checkbox').property('checked', false)|
|.text|Update the text content|d3.select('div.title').text('My new book')|
|.html|Change the html content|d3.select('.legend').html('<span>0 - 10</span>')|
- Exercise: [D3 - Selections](http://blockbuilder.org/mukhtyar/e2e2d41f9828ea4459e0161878cf8f40)
???
Methods as dual purpose getters and setters is something you will see and use a lot. selection.attr is for working with element attributes. When calling selection.attr(name) the method will act as a getter and return the name attribute's value. selection.attr(name, value) will cause the method to act as a setter, assigning value to the name attribute.
---
## Core Concepts - Selections (contd.)
- Append/insert elements to elements to existing selections
- .bold[selection.append(type) && selection.insert(type[, before])]
```javascript
d3.select('body').append('p');
```
- Remove items from selections
- .bold[selection.remove()]
```javascript
d3.select('body').remove('p');
```
- Handle events
- .bold[selection.on()]
```javascript
d3.selectAll('button').on('click', handleButtonClicks);
d3.select('body').on('mousemover', trackMousePosition);
```
- Read documentation [d3-selection](https://github.com/d3/d3-selection)
---
## Core Concepts - Data Binding
- Real power of D3 is joining data and selections
- .bold[selection.data()] - accepts an array of data items and joins them to the DOM elements in the selection
- Exercise: [D3 - Data Binding 1](http://blockbuilder.org/mukhtyar/1e6652ca51b77c78e4567874137ebc01)
- Exercise with SVG: [D3 - Data Binding 2](http://blockbuilder.org/mukhtyar/182862bdc1cce6f3c19cf5bc657c209c)
???
But when it comes to controlling a document or visualization based on underlying data, D3's power and expressiveness are truly unparalleled.
The fundamental API for this is selection.data which accepts an array of data items and joins them to the DOM elements in the selection.
---
## Core Concepts - Data Binding (contd.)
- What if we have more data than existing elements? Enter...
- Exercise: [D3 - Enter Selection](http://blockbuilder.org/mukhtyar/56405be0691ebeff058ce91804cab001)
- ENTER, UPDATE and EXIT SELECTIONS
- Mike Bostock - [Thinking with Joins](https://bost.ocks.org/mike/join/)
- Mike Bostock - [General Update Pattern examples](https://bl.ocks.org/mbostock/3808218)
- Also [D3 In Depth book](http://d3indepth.com/) (in progress) is great!
---
class: img100
## Core Concepts - Data Binding (contd.)
---
## Core Concepts - Data Binding (contd.)
- Use with dynamic data
- Exercise: [D3 - Enter, Update, Exit](http://blockbuilder.org/mukhtyar/2eaf29426bdc09a328fd7666a01310df)
- Use with data from Cal-Adapt API
- Exercise: [D3 - Get Data from API](http://blockbuilder.org/mukhtyar/61d40e36f4c187c5a36207fda5a00e6d)
---
class: img100
## Core Concepts - Data Binding (contd.)
---
## Core Concepts - Scales
- Mapping: An operation that associates each element of a given set (the domain) with one or more elements of a second set (range)
- Any (computer) visualization problem can be described as a mapping problem.
- Documentation - [d3.scales](https://github.com/d3/d3-scale)
???
another one of D3’s fundamental concepts is devoted entirely to transformations. Data visualization is ultimately about transformation. Quarterly revenue becomes column height, the winning political party becomes a state's fill color on a map, and on and on. In D3 these transformations are done using scales, and they're another one of D3's core concepts.
Linear scales are the most straightforward as they directly translate values from one context to another.
Calling d3.scaleLinear() will instantiate a new linear scale. We then call two methods on the scale, passing each of them a two-element array representing the minimum and maximum values. The domain() method is used to specify the possible input values while range() specifies the corresponding output values. The return value of this code is a scale function named percentToDecimal. This function will accept a percentage value and return the corresponding decimal.
---
## Core Concepts - Scales (contd.)
- Mapping from data attributes (domain) to display (range)
- e.g. date -> x value, data value -> opacity, data value -> y value
```javascript
d3.scaleLinear()
.domain([min, max]) // input
.range([min, max]); // output
```
- Linear scale
```javascript
const percentToDecimal = d3Scale.scaleLinear()
.domain([0, 100])
.range([0, 1])
percentToDecimal(0) // 0
percentToDecimal(50) // 0.5
percentToDecimal(100) // 1
```
- Exercise - [d3.scales](https://github.com/d3/d3-scale)
---
## Core Concepts - Axes
- Human readable reference for scales
- Documentation - [d3-axis](https://github.com/d3/d3-axis)
- Exercise - [Bar Chart](http://blockbuilder.org/mukhtyar/054ffe48763e1c28448ae0d032e12ffe)
- Exercise - [Bar Chart using Cal-Adapt data](http://blockbuilder.org/mukhtyar/f50577e4bad9f522ec8bc0d3500fa1bc)
---
## D3 - Other concepts
- Lots of convenience functions for loading and manipulating data, working with arrays
- Shapes - .bold[d3.line()], .bold[d3.circle()], .bold[d3.pie()], .bold[d3.arc()]
- Colors
- Manipulate structure of your data - d3.nest()
- D3 [API](https://github.com/d3/d3/blob/master/API.md) Reference
- Writing reusable components
- A good place to start is by reading [Towards Resuable Charts](https://bost.ocks.org/mike/chart/) where Mike Bostock proposes a convention for creating reusable charts.
- [This](https://bocoup.com/weblog/reusability-with-d3) blog post and [this book](http://backstopmedia.booktype.pro/developing-a-d3js-edge/1-getting-started/) do a great job on clarifying the resuable charts api with examples and explanations.
---
## Other options for using D3
- Rickshaw, Highcharts, NVD3 are libraries built on top of D3
- D3 and [Crossfilter](http://square.github.io/crossfilter/) (Fast Multidimensional Filtering for Coordinated Views)
- Open-source tools binding D3 to R, Python
- [Vega](https://github.com/vega/vega/wiki/Vega-and-D3) - higher-level visualization specification language on top of D3
---
class: center
<br/>
# Web Mapping
<br/>
---
## What is a Web Map?
- Digital map that works in a web browser (as opposed to PDF/Image files)
- Interactive
- Developed using web application technologies
– Not specialized mapping software
---
## A Quick Recap of Web Mapping History
- 1996 MapQuest
---
## A Quick Recap of Web Mapping History (contd.)
- 1997 - 2004
- MapInfo, AOL buys MapQuest
- GPS Selective Availability ends (2000)
- Microsoft MapPoint
- Open Street Map (OSM) founded in UK (2004)
- NASA releases WorldWind
- Yahoo! Maps
- Google buys Where2, Keyhold & ZipDash
---
## A Quick Recap of Web Mapping History (contd.)
- 2005 - Google Maps
---
## A Quick Recap of Web Mapping History (contd.)
- 2005 - First Google Maps Mashup
---
## Technological Breakthroughs
- Slicing a square map into smaller square image tiles
– Web Mercator projection
- Slippy maps with smooth zoom and pan
– Web development techniques (AJAX) used for requesting data (map tiles) from server asynchronously (in the background)
- Open API
- No plugins, seamless user experience
- Mapping as a service
---