Updated Basic Usage of Manipulation Functions #3360

nathanrboyer · 2023-07-18T19:51:10Z

Replaces PR #2907 since it was too out of sync with the main branch.

docs/src/man/basics.md

nathanrboyer · 2023-09-15T13:44:18Z

Friendly bump 🙂

bkamins · 2023-09-15T17:20:36Z

Agreed. But first we need to make a decision on #3361. I will have a look at it and comment there.

bkamins · 2023-09-25T21:18:52Z

docs/src/man/basics.md

+| ------------ | -------------------------------- | -------------------------------------------- | ------------------------------------------------- |
+| `transform`  | Creates a new data frame.        | Retains both source and manipulated columns. | Retains same number of rows as source data frame. |
+| `transform!` | Modifies an existing data frame. | Retains both source and manipulated columns. | Retains same number of rows as source data frame. |
+| `select`     | Creates a new data frame.        | Retains only manipulated columns.            | Retains same number of rows as source data frame. |


"manipulated" or "created"?

bkamins · 2023-09-25T21:19:46Z

docs/src/man/basics.md

+| `transform!` | Modifies an existing data frame. | Retains both source and manipulated columns. | Retains same number of rows as source data frame. |
+| `select`     | Creates a new data frame.        | Retains only manipulated columns.            | Retains same number of rows as source data frame. |
+| `select!`    | Modifies an existing data frame. | Retains only manipulated columns.            | Retains same number of rows as source data frame. |
+| `subset`     | Creates a new data frame.        | Retains only source columns.                 | Number of rows is determined by the manipulation. |


maybe better say that by condition (to differentiate it from combine)

docs/src/man/basics.md

bkamins · 2023-09-25T21:24:41Z

docs/src/man/basics.md

+### Constructing Operation Pairs
+All of the functions above use the same syntax which is commonly
+`manipulation_function(dataframe, operation)`.
+The `operation` argument is a `Pair` which defines the


maybe add that Pair is constructed using =>?

Also - it does not have to be the pair, so maybe say it is usually a pair?
(your first example is not a pair below)

docs/src/man/basics.md

bkamins · 2023-09-25T21:28:31Z

docs/src/man/basics.md

-These rules are typically called transformation mini-language.
-
-Let us move to the examples of application of these rules
+## Basic Usage of Manipulation Functions


A larger question - maybe create a separate page for this tutorial?

I think that would be better. Maybe "Manipulation Functions" under "User Guide" before "Split-apply-combine"?

As commented below - I would put it as a "top level" with a name something along "A gentle introduction to manipulation functions" (so that we clearly signal that this material is less formal than the rest of the manual).

What would you think of making the new Top Level section something like "Beginner's Guide" or "User's Guide for Beginners" and then placing "Manipulation Functions" at a second level under that? I'm not volunteering to rewrite the entire User's Guide, but it could leave room for others to add similar "gentle" content to the documentation. It would also make the sidebar look cleaner by splitting up the current long name.

Makes sense. The other section that could go there is https://dataframes.juliadata.org/stable/man/basics/ as it has the same objective.

Oh, but Basics contains the existing "Basic Usage of Manipulation Functions". I don't know how to differentiate it from this new section if they live next to each other.

I initially intended to just clarify some topics within that section, but now the scope has grown.

I can maybe overwrite that section if I add these topics that I don't currently cover:

"Note that this time we use string column selectors because some of the column names have spaces in them."

"The benefit of select or combine over indexing is that it is easier to get the union of several column selectors."

"It is important to note that select always returns a data frame, even if a single column selected as opposed to indexing syntax."

"By default select copies columns of a passed source data frame. In order to avoid copying, pass the copycols=false keyword argument."

The other sections under Basics use the German dataset, but I think it is easier to understand what is going on with smaller data frames where you know all the data values.

I can maybe overwrite that section

Yes - I think it is OK just to expand that section (especially that it is top-level now already)

I think it is easier to understand what is going on with smaller data frames

Agreed. just please use different variable names than these already used there so that using different dataframes does not lead to confusion.

Thank you! (sorry for so many comments, but - unfortunately - writing documentation is hard)

Done (sort of). I did not use new variables names though. I had been and continued to just overwrite the definition of df. My data frames are so small and frequent that coming up with a new name each time would be a pain.

bkamins · 2023-09-25T21:29:40Z

docs/src/man/basics.md

+!!! Note
+      The Julia parser sometimes prevents `:` from being used by itself.
+      `ERROR: syntax: whitespace not allowed after ":" used for quoting`
+      means your `:` must be wrapped in either `(:)` or `Cols(:)`


I typically recommend All() which is easy to understand I think.

docs/src/man/basics.md

nathanrboyer · 2023-09-28T20:33:19Z

Thank you for the review. I believe I addressed all your comments, but I left it to you to determine if satisfactory and "Resolve" them.

bkamins · 2023-09-30T16:52:37Z

documenter fails - have you checked why?

bkamins · 2023-09-30T16:54:51Z

docs/src/man/basics.md

-These rules are typically called transformation mini-language.
-
-Let us move to the examples of application of these rules
+These functions and their methods are explained in more detail in the section


Maybe add that not only more detail, but with more "slow paced" and informal approach :).

bkamins · 2023-09-30T16:55:29Z

docs/src/man/manipulation_functions.md

@@ -0,0 +1,1345 @@
+# Data Frame Manipulation Functions
+
+The seven functions below can be used to manipulate data frames


Also here, as I have commented - maybe add a sentence at the top explaining the teaching approach of this material. Thank you!

nathanrboyer · 2023-10-02T19:29:13Z

documenter fails - have you checked why?

I've never used Documenter.jl before. I just tried to edit the make.jl and index.md files by copying what was already there.

nathanrboyer · 2023-10-02T21:06:14Z

I don't know why the cross references aren't working. It looks to me like I am doing it right. https://documenter.juliadocs.org/dev/man/syntax/#@ref-link

nathanrboyer · 2023-10-13T21:10:55Z

Need to see if documenter tests pass this time. There is also a subsection I would like at the end on performance, but I am hoping someone else can write it, even if that is as a separate PR.

bkamins · 2023-10-14T10:03:55Z

I think that the documenter error is due to the fact that your references are on level three, i.e. ### header. And these headers are not included in TOC.

@mortenpi - how could @nathanrboyer reference e.g. this section
https://github.com/JuliaData/DataFrames.jl/pull/3360/files#diff-80725fcadb5509a5fb0986e534b25ae5c65bab29c2903e8319c67c7a42143ffaR2615
in this
https://github.com/JuliaData/DataFrames.jl/pull/3360/files#diff-80725fcadb5509a5fb0986e534b25ae5c65bab29c2903e8319c67c7a42143ffaR1642
position? Thank you!

docs/src/man/basics.md

mortenpi · 2023-12-06T23:23:20Z

@mortenpi - how could @nathanrboyer reference e.g. this section

Sorry, missed this earlier. This actually looks like a bug. The fact that it's level 3 heading shouldn't matter. As a workaround, you might be able to use @id-s in the headers though.

Initial commit

5785c72

nathanrboyer changed the title ~~Initial commit~~ Updated Basic Usage of Manipulation Functions Jul 18, 2023

nathanrboyer added 2 commits July 19, 2023 11:32

assumes requested method is added

f5cc15f

Typo: missing :z

7c3db8a

bkamins reviewed Jul 25, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

nathanrboyer added 3 commits July 25, 2023 12:10

added subset(df, source_column_selector)

27d7e32

Added italics

be5fa9e

Moved note to main text

b1b3bab

nathanrboyer commented Aug 2, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

nathanrboyer commented Aug 2, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

nathanrboyer and others added 5 commits August 2, 2023 15:02

Added error example and removed ° symbol

da6607d

Moved Additional Resources to the end and cleaned

0c47d10

Capitalized Boolean

6f5dfc5

Merge branch 'main' into nb/manipulation_function_basics

e47cbdf

Removed extra space character

886d998

nathanrboyer added 3 commits September 18, 2023 12:02

Change function broadcasting to avoid old language

cabd73f

Made consistent with current proposal JuliaData#3361

2e9d2af

Change α to apple and make consistent with JuliaData#3380

b0777b1

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

bkamins reviewed Sep 26, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

First round review corrections

a111ef8

nathanrboyer added 3 commits September 29, 2023 11:07

Move to its own section

ce55607

Add new file to make and index

46363d9

Rewrite Basics.md conclusion

cd4c539

bkamins reviewed Sep 30, 2023

View reviewed changes

Review Edits Round 2

d70af83

nathanrboyer and others added 3 commits October 2, 2023 15:30

Merge branch 'main' into nb/manipulation_function_basics

e43346e

Fix reference?

6377441

maybe fix documenter?

6e7ed84

nathanrboyer added 3 commits October 5, 2023 12:29

make h function require broadcasting

d2d3de8

Fix existing typos in basics.md

0bdfc44

Move back to basics.md and add comparison

72d87d2

nathanrboyer added 2 commits October 13, 2023 17:17

Add more comments

043605a

Add link to @with macro

26b503e

bkamins reviewed Oct 14, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

bkamins reviewed Oct 14, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

nathanrboyer and others added 3 commits October 14, 2023 21:42

Delete redundant expression

679f65f

Clean up new section and delete with reference

79a1171

Merge branch 'main' into nb/manipulation_function_basics

621f253

Merge branch 'main' into nb/manipulation_function_basics

7614fc3

nathanrboyer mentioned this pull request May 20, 2024

Cannot find level-three cross-references JuliaDocs/Documenter.jl#2507

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated Basic Usage of Manipulation Functions #3360

Updated Basic Usage of Manipulation Functions #3360

nathanrboyer commented Jul 18, 2023 •

edited

nathanrboyer commented Sep 15, 2023

bkamins commented Sep 15, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

nathanrboyer Sep 28, 2023

bkamins Sep 30, 2023

nathanrboyer Oct 4, 2023

bkamins Oct 9, 2023

nathanrboyer Oct 9, 2023

bkamins Oct 9, 2023

nathanrboyer Oct 13, 2023

bkamins Sep 25, 2023

nathanrboyer commented Sep 28, 2023

bkamins commented Sep 30, 2023

bkamins Sep 30, 2023

bkamins Sep 30, 2023

nathanrboyer commented Oct 2, 2023

nathanrboyer commented Oct 2, 2023

nathanrboyer commented Oct 13, 2023

bkamins commented Oct 14, 2023

mortenpi commented Dec 6, 2023

		@@ -0,0 +1,1345 @@
		# Data Frame Manipulation Functions

		The seven functions below can be used to manipulate data frames

Updated Basic Usage of Manipulation Functions #3360

Are you sure you want to change the base?

Updated Basic Usage of Manipulation Functions #3360

Conversation

nathanrboyer commented Jul 18, 2023 • edited

nathanrboyer commented Sep 15, 2023

bkamins commented Sep 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nathanrboyer commented Sep 28, 2023

bkamins commented Sep 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nathanrboyer commented Oct 2, 2023

nathanrboyer commented Oct 2, 2023

nathanrboyer commented Oct 13, 2023

bkamins commented Oct 14, 2023

mortenpi commented Dec 6, 2023

nathanrboyer commented Jul 18, 2023 •

edited