Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: warehouse #23714

Merged
merged 127 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
b15f53a
add event collection ui, queries, routing
jordienr Feb 22, 2024
23288e1
Merge branch 'master' into feat/event-collections
jordienr Mar 4, 2024
0464606
fix alignment issue
jordienr Mar 4, 2024
0fb889c
renaming
jordienr Mar 4, 2024
f85486f
wip
jordienr Mar 5, 2024
7379eaa
feat: initial data hooks and integration with feature flagging
Ziinc Mar 7, 2024
8f3a35e
Merge branch 'master' into feat/event-collections
jordienr Mar 20, 2024
7210f1a
Merge branch 'feat/warehouse-integration' into feat/event-collections
jordienr Mar 20, 2024
4d644c9
update with naming, correct mocks, etc
jordienr Mar 25, 2024
58f3439
feat: add source crud integration
Ziinc Mar 27, 2024
f4fbf26
mock data, fix form, add motion to drawer
jordienr Mar 27, 2024
dc43b4f
mock data, fix form, add motion to drawer
jordienr Mar 27, 2024
456f03b
feat: optimize warehouse fetch behaviour
Ziinc Mar 28, 2024
f404e00
feat: add in valid query
Ziinc Mar 28, 2024
c4f4058
Merge branch 'feat/event-collections' of github.com:supabase/supabase…
Ziinc Mar 28, 2024
320cde0
chore: ermove collection mocks
Ziinc Mar 28, 2024
3e0d505
feat: optimize tenant fetching
Ziinc Mar 28, 2024
b9dea8c
Merge branch 'master' into feat/event-collections
jordienr Apr 12, 2024
968eac3
cleanup + update openapi client + fix fetch calls
jordienr Apr 16, 2024
9d01097
cleanup imporst
jordienr Apr 16, 2024
48f48bb
on delete success, redirect to main logs page
jordienr Apr 16, 2024
eae72d8
fix dropdown event propagating to link and opening page
jordienr Apr 16, 2024
66c0a4b
Refactor code to use useParams and useRouter in CreateWarehouseCollec…
jordienr Apr 16, 2024
3a7885f
clean warehouse query
jordienr Apr 16, 2024
014e8a8
Merge branch 'master' of https://github.com/supabase/supabase into fe…
jordienr Apr 18, 2024
b2d51ab
Merge branch 'master' into feat/event-collections
jordienr Apr 22, 2024
0c804f5
warehouse access tokens create, list
jordienr Apr 23, 2024
162b0ad
add warehouse access token revokal
jordienr Apr 23, 2024
6b4d5b0
fix access tokens table, fix access tokens revoke feature
jordienr Apr 24, 2024
e7ffcbe
format date in access tokens table
jordienr Apr 24, 2024
1b80230
fix some state issues and warnings in access tokens forms
jordienr Apr 25, 2024
3e916ee
merge master
jordienr Apr 25, 2024
5d1772d
warehouse/feat: introduction when creating event collections (#23244)
Ziinc Apr 25, 2024
0b631e4
warehouse/feat: rename headers and nav items (#23243)
Ziinc Apr 25, 2024
9a8740b
ui fixes, rm deprecated icons
jordienr Apr 29, 2024
8c7dc42
render table with events
jordienr Apr 29, 2024
1f587ee
fix typeerror results arr
jordienr Apr 29, 2024
8e7f708
add pagination
jordienr Apr 30, 2024
e0a1a2e
some ui fixes
jordienr Apr 30, 2024
8fa92ba
fix loading states, format dates, test connection dialog
jordienr May 2, 2024
3428482
Merge branch 'master' into feat/event-collections
jordienr May 2, 2024
9aebca5
Merge branch 'master' of https://github.com/supabase/supabase into fe…
jordienr May 2, 2024
a6b0a5a
patch type errors
jordienr May 2, 2024
806571a
Fix access token selection in TestCollectionDialog and update column …
jordienr May 2, 2024
da76eac
rm unused querytype
jordienr May 2, 2024
e884e6a
rm unused querytype in logtable
jordienr May 2, 2024
d115bb8
patch type errors in warhouse collection mutations
jordienr May 2, 2024
11c19fd
rm unused component
jordienr May 2, 2024
c366417
mr unused import
jordienr May 2, 2024
beacd22
show connection dialog in staging
jordienr May 3, 2024
ab81838
fix Run btn now showing up
jordienr May 8, 2024
9abbe0b
fix undef argument
jordienr May 8, 2024
50f74b0
show pagination only when there are results
jordienr May 8, 2024
e5fc218
merge master, update ui in test connection dialog
jordienr May 10, 2024
32a4306
fix issues in connection dialog, improve ux
jordienr May 10, 2024
b036e94
Update apps/studio/components/interfaces/DataWarehouse/CreateWarehous…
jordienr May 15, 2024
2268ca0
Update apps/studio/components/interfaces/DataWarehouse/CreateWarehous…
jordienr May 15, 2024
e9a442a
fix conflicts, fix pagination bug
jordienr May 16, 2024
0af71f2
fix infinite render loop, adapt component to render log metadata on l…
jordienr May 16, 2024
1dbb78c
rm unused import, prevent ui from breaking when some data may be miss…
jordienr May 16, 2024
1211586
update WAT create mutation to use template
jordienr May 17, 2024
3d04d07
follow mutation template in delete access token mutation
jordienr May 17, 2024
c93a872
refactor naming, improve code in some areas
jordienr May 20, 2024
db120f9
merge master
jordienr May 20, 2024
d9e109d
rm unused mocks for msw
jordienr May 20, 2024
3206a68
skip tests - this will get fixed on the vitest PR
jordienr May 20, 2024
047bde5
Update apps/studio/components/interfaces/Settings/Logs/LogsQueryPanel…
jordienr May 20, 2024
ba0fac3
fix loading compo
jordienr May 20, 2024
af878f4
rm unused import
jordienr May 20, 2024
52789ce
use correct alert title
jordienr May 20, 2024
6cbfb0d
rm unnecessary div
jordienr May 20, 2024
07a7d34
fix layout in projectpaused alert
jordienr May 20, 2024
56d8ff1
fix type err, refactor to new method
jordienr May 20, 2024
91b64af
rm observer
jordienr May 20, 2024
930bf65
rm observer
jordienr May 20, 2024
0e8ae89
refactor warehouse collections update
jordienr May 20, 2024
5613e51
use correct get method, comment out so it doesnt err
jordienr May 20, 2024
99d6a79
use correct method
jordienr May 20, 2024
8df58d6
fix formatting issues
jordienr May 20, 2024
4c9439a
ref imports
jordienr May 21, 2024
021f2f9
ref imports
jordienr May 21, 2024
dcf88de
clean up imports
jordienr May 21, 2024
bad0ada
clean imports
jordienr May 21, 2024
abf5604
clean imports
jordienr May 21, 2024
0dfa87b
rename props
jordienr May 21, 2024
6999423
destructure props
jordienr May 21, 2024
52cd3de
cleanup imports
jordienr May 21, 2024
aca85cb
Update apps/studio/components/interfaces/DataWarehouse/CreateWarehous…
jordienr May 21, 2024
f3f2ac1
ref scale class
jordienr May 21, 2024
407a7c7
cleanup divs
jordienr May 21, 2024
ce050fd
Update apps/studio/components/interfaces/DataWarehouse/TestCollection…
jordienr May 21, 2024
dfc4f16
refactor to rhf
jordienr May 21, 2024
072a4b4
Merge branch 'feat/warehouse' of https://github.com/supabase/supabase…
jordienr May 21, 2024
c57a241
reuse types, cleanup, refactor to react query hooks instead of el events
jordienr May 21, 2024
db0f558
refactor table component to handle empty state
jordienr May 21, 2024
698fcd5
Update TestCollectionDialog to use unique keys for SelectItem components
jordienr May 21, 2024
c3608e1
Update TestCollectionDialog to use unique keys for SelectItem components
jordienr May 21, 2024
5fb6efe
Update TestCollectionDialog to use unique keys for SelectItem components
jordienr May 21, 2024
74e649c
fix empty state ui
jordienr May 21, 2024
d9b9451
use destructured queries
jordienr May 21, 2024
f74d3ad
rename query types
jordienr May 21, 2024
e4c9565
Update apps/studio/components/interfaces/DataWarehouse/WarehouseAcces…
jordienr May 21, 2024
a03ba5d
chore: Refactor WarehouseCollectionDetail component
jordienr May 21, 2024
e42084d
rm unused imports
jordienr May 21, 2024
570a0da
Merge branch 'feat/warehouse' of https://github.com/supabase/supabase…
jordienr May 21, 2024
7fb881e
rename loadolder
jordienr May 22, 2024
011a1af
rm typcast
jordienr May 22, 2024
a66e4f5
rhf in create token
jordienr May 23, 2024
d0f2c8c
improve err handling
jordienr May 23, 2024
eb902ae
refactor to rhf, refactor queries, use cn utils, cleanup
jordienr May 23, 2024
0ec8b9c
fix loading logselection
jordienr May 23, 2024
7046eb4
add correct type to warehouse access tokens query
jordienr May 23, 2024
b558440
use correct types in err & clean unused types
jordienr May 23, 2024
e2bfeed
toggle token in curl exampel
jordienr May 24, 2024
707db58
use product empty state component in warehouse
jordienr May 27, 2024
655c7c0
add client side validation of collection name, temp fix
jordienr May 27, 2024
66bc0d7
add New badge
jordienr May 27, 2024
bad67cb
update testcolldialog
jordienr May 28, 2024
32f6d85
cleanup unnecessary div and rm double border
jordienr May 28, 2024
4322735
code style fixes
jordienr May 28, 2024
d79d6fd
handle err
jordienr May 28, 2024
e4561f2
code style fix
jordienr May 28, 2024
187eddc
add correct type
jordienr May 28, 2024
ee8c213
add correct err type
jordienr May 28, 2024
2d993ab
fix type error
jordienr May 28, 2024
4d47d53
fix type err
jordienr May 28, 2024
45c24a2
fix missing type
jordienr May 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import { set } from 'lodash'
import React from 'react'
jordienr marked this conversation as resolved.
Show resolved Hide resolved
import { Button, Input, Modal } from 'ui'
import { FormItemLayout } from 'ui-patterns/form/FormItemLayout/FormItemLayout'

type Props = {
onSubmit: (values: { description: string }) => Promise<void>
jordienr marked this conversation as resolved.
Show resolved Hide resolved
}

const CreateWarehouseAccessToken = (props: Props) => {
jordienr marked this conversation as resolved.
Show resolved Hide resolved
const [open, setOpen] = React.useState(false)
const [loading, setLoading] = React.useState(false)
const [description, setDescription] = React.useState('')
jordienr marked this conversation as resolved.
Show resolved Hide resolved

async function onConfirm() {
setLoading(true)

await props.onSubmit({
description,
})
setDescription('')
setLoading(false)
setOpen(false)
}

return (
<>
<Button type="outline" onClick={() => setOpen(true)}>
Create access token
</Button>
<Modal
size="medium"
onCancel={() => {
setOpen(false)
}}
header="Create access token"
visible={open}
alignFooter="right"
loading={loading}
onConfirm={onConfirm}
>
<Modal.Content className="py-4">
<form
onSubmit={async (e) => {
e.preventDefault()
onConfirm()
}}
>
<FormItemLayout
label="Description"
description="A short description identifying this access token."
isReactForm={false}
>
<Input
placeholder="Description"
name="description"
id="description"
onChange={(e) => setDescription(e.target.value)}
value={description}
/>
</FormItemLayout>
</form>
</Modal.Content>
</Modal>
</>
)
}

export default CreateWarehouseAccessToken
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
import { useParams } from 'common'
import { useCreateCollection } from 'data/analytics/warehouse-collections-create-mutation'
jordienr marked this conversation as resolved.
Show resolved Hide resolved
import { useRouter } from 'next/router'
import React from 'react'
import toast from 'react-hot-toast'
import { Button, IconPlus, Input, Modal } from 'ui'

type Props = {}
jordienr marked this conversation as resolved.
Show resolved Hide resolved

export const CreateWarehouseCollectionModal = (props: Props) => {
const [isOpen, setIsOpen] = React.useState(false)
const router = useRouter()
const { ref } = useParams()

const {
mutateAsync: createCollection,
jordienr marked this conversation as resolved.
Show resolved Hide resolved
isLoading,
data: newCollection,
} = useCreateCollection({
projectRef: ref || 'default',
jordienr marked this conversation as resolved.
Show resolved Hide resolved
onSuccess: (data) => {
router.push(`/project/${ref}/logs/collections/${data.data?.token}`)
},
})

return (
<>
<Button
type="default"
className="justify-start flex-grow w-full"
icon={<IconPlus size="tiny" />}
jordienr marked this conversation as resolved.
Show resolved Hide resolved
onClick={() => {
setIsOpen(!isOpen)
}}
jordienr marked this conversation as resolved.
Show resolved Hide resolved
>
New collection
</Button>
<Modal
size="medium"
onCancel={() => setIsOpen(!isOpen)}
header="Create an event collection"
visible={isOpen}
hideFooter
>
<form
jordienr marked this conversation as resolved.
Show resolved Hide resolved
onSubmit={async (e) => {
e.preventDefault()
try {
const formData = new FormData(e.target as HTMLFormElement)
const values = {
name: formData.get('name') as string,
}
await createCollection(values)
toast.success(`Collection ${values.name} created`)
} catch (error) {
console.error(error)
toast.error(`Failed to create collection. Check the console for more details.`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an aside: is this really the best ux? If we're asking users to check the console, might as well display the error message in a modal or textbox or something. I'd opt for a simple stringify of the error and chuck it in an alert wrapped with a <pre> so that people can copy-paste the error out to support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this error handling shouldn't be here - it should be in createCollection itself, we won't need the try catch as well if we use mutate instead of mutateAsync

the toast.success should be in the onSuccess callback of the RQ mutation, and the error handling should be in the onError callback of the RQ mutation as well (all RQ mutations would have a default fallback error handler as well, so unless this component needs a specific error handling behaviour, we can leave it)

but yes - always surface the error message, don't let the user think 💪

} finally {
setIsOpen(false)
}
}}
>
<div className="py-4">
<Modal.Content>
<p className="pb-5 text-scale-1100 text-sm">
jordienr marked this conversation as resolved.
Show resolved Hide resolved
An event collection stores generic timeseries events and metadata in
Supabase-managed analytics infrastructure. Events can be then be queried using SQL,
without impacting transactional workloads.
</p>
<div className="space-y-6">
<Input
required
layout="horizontal"
label="Collection name"
id="name"
name="name"
autoComplete="off"
/>
</div>
</Modal.Content>
</div>
<div className="py-3 border-t bg-surface-100">
jordienr marked this conversation as resolved.
Show resolved Hide resolved
<Modal.Content>
<div className="flex items-center justify-end gap-2">
<Button size="tiny" type="default" onClick={() => setIsOpen(!isOpen)}>
Cancel
</Button>
<Button size="tiny" loading={isLoading} disabled={isLoading} htmlType="submit">
Create table
</Button>
</div>
</Modal.Content>
</div>
</form>
</Modal>
</>
)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
import { SelectContent, SelectItem, SelectTrigger, Select } from '@ui/components/shadcn/ui/select'
import { useWarehouseCollectionsQuery } from 'data/analytics/warehouse-collections-query'
import { useRouter } from 'next/router'
import { useEffect, useState } from 'react'
import {
Button,
CodeBlock,
Dialog,
DialogContent,
DialogDescription,
DialogHeader,
DialogSection,
DialogSectionSeparator,
DialogTitle,
DialogTrigger,
Input,
TooltipContent_Shadcn_,
TooltipTrigger_Shadcn_,
Tooltip_Shadcn_,
} from 'ui'
import { FormItemLayout } from 'ui-patterns/form/FormItemLayout/FormItemLayout'

export function TestCollectionDialog({
accessTokens,
collectionToken,
collections,
}: {
accessTokens: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these can be moved to its own types for clarity

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed ^ either that or if they are coming directly from the API, use the types from the codegen

id: string
token: string
description?: string
}[]
collections: {
id: number
token: string
name: string
}[]
collectionToken: string
projectRef: string
}) {
const BASE_WAREHOUSE_URL = `https://api.warehouse.tech/api/events`
const [testAccessToken, setTestAccessToken] = useState('')
const [selectedCollection, setSelectedCollection] = useState(collectionToken || '')

useEffect(() => {
setSelectedCollection(collectionToken)
}, [collectionToken])

useEffect(() => {
if (accessTokens.length > 0) {
setTestAccessToken(accessTokens[0].token)
}
}, [accessTokens])

const selectedAccessToken = accessTokens.find((token) => token.token === testAccessToken)
const selectedCollectionName = collections.find((col) => col.token === selectedCollection)?.name

return (
<Dialog>
<Tooltip_Shadcn_>
<TooltipTrigger_Shadcn_ asChild>
<DialogTrigger asChild>
<Button disabled={accessTokens.length === 0} type="outline">
Connect
</Button>
</DialogTrigger>
</TooltipTrigger_Shadcn_>
{accessTokens.length === 0 && (
<TooltipContent_Shadcn_>
Create an access token to connect to your collection
</TooltipContent_Shadcn_>
)}
</Tooltip_Shadcn_>

<DialogContent>
<DialogHeader>
<DialogTitle>Send events to this collection</DialogTitle>
<DialogDescription>
Use the following curl command to send events to this collection
jordienr marked this conversation as resolved.
Show resolved Hide resolved
</DialogDescription>
</DialogHeader>
<DialogSectionSeparator />
<DialogSection className="flex flex-col gap-4 overflow-auto">
<div className="flex gap-2 *:flex-1">
<FormItemLayout label="Collection" isReactForm={false}>
<Select value={selectedCollection} onValueChange={setSelectedCollection}>
<SelectTrigger>
<span className="text-ellipsis">{selectedCollectionName || 'Collection'}</span>
jordienr marked this conversation as resolved.
Show resolved Hide resolved
</SelectTrigger>
<SelectContent>
{collections.map((col) => (
<SelectItem key={col.id + '-collection'} value={col.token}>
jordienr marked this conversation as resolved.
Show resolved Hide resolved
{col.name || 'No name'}
jordienr marked this conversation as resolved.
Show resolved Hide resolved
</SelectItem>
))}
</SelectContent>
</Select>
</FormItemLayout>
<FormItemLayout label="Token" isReactForm={false}>
<Select value={testAccessToken} onValueChange={setTestAccessToken}>
<SelectTrigger>
<span className="text-ellipsis">
{selectedAccessToken?.description || 'Access token'}
</span>
</SelectTrigger>
<SelectContent>
{accessTokens?.map((token: any) => (
<SelectItem key={token.id + '-token'} value={token.token}>
{token.description || 'No description'}
</SelectItem>
))}
</SelectContent>
</Select>
</FormItemLayout>
</div>

<FormItemLayout label="Ingest URL" isReactForm={false}>
<Input
className="font-mono tracking-tighter"
value={BASE_WAREHOUSE_URL}
readOnly
copy
/>
</FormItemLayout>

<div>
jordienr marked this conversation as resolved.
Show resolved Hide resolved
<CodeBlock className="p-1 language-bash prose" language="bash">
{`curl -X "POST" "${BASE_WAREHOUSE_URL}?source=${selectedCollection}" \\
jordienr marked this conversation as resolved.
Show resolved Hide resolved
-H 'Content-Type: application/json' \\
-H 'X-API-KEY: ${testAccessToken || 'ACCESS_TOKEN'}' \\
-d $'{
"event_message": "Test event message",
"metadata": {
"ip_address": "100.100.100.100",
"request_method": "POST",
"custom_user_data": {
"foo": "bar"
},
"datacenter": "aws",
"request_headers": {
"connection": "close",
"user_agent": "chrome"
}
}
}'
`}
</CodeBlock>
</div>
</DialogSection>
</DialogContent>
</Dialog>
)
}