Short

Preview

Get s/ Chrome extension

Install it from Chrome Web Store or build it from source

Dependent Projects

• app: Reusable framework for Go apps & command line tools.
• kgs: Offline unique key generation service.
• toggle: Dynamic system behavior controller.

Table of Contents

1. Getting Started
   1. Accessing the source code
   2. Prerequisites
   3. Create reCAPTCHA account
   4. Create Github OAuth application
   5. Create Facebook Application
   6. Create Google OAuth Client ID
   7. Backend
   8. Frontend
2. System Design
   1. App Level Architecture
   2. Service Level Archtecture
   3. Object Oriented Design
   4. Dependency Injection
   5. Database Modeling
   6. Feature Toggle
   7. Search Engine Optimization
   8. Social Media Summary Card
3. Testing
   1. The Importance Of Automation
   2. Testing Strategy
   3. Unit Testing
   4. Integration Testing
   5. Component Testing
   6. Contract Testing
   7. End To End Testing
   8. The Test Pyramid
4. Deployment
   1. Kubernetes
   2. Staging
   3. Production
5. Tools We Use
6. Contributing
7. Author
8. License

Getting Started

Accessing the source code

git clone https://github.com/byliuyang/short.git

Prerequisites

• Go v1.13.1
• Node.js v12.12.0
• Yarn v1.19.1
• Postgresql v12.0 ( or use ElephantSQL instead )

Create reCAPTCHA account

1. Sign up at ReCAPTCHA with the following configurations:

   Field	Value
   Label	Short
   reCAPTCHA type	reCAPTCHAv3
   Domains	localhost

   

2. Open settings. Copy SITE KEY and SECRET KEY.

   

   

3. Replace the value of RECAPTCHA_SECRET in the backend/.env file with SECRET KEY.

4. Replace the value of REACT_APP_RECAPTCHA_SITE_KEY in frontend/.env.development file with SITE KEY.

Create Github OAuth application

1. Create a new OAuth app at Github Developers with the following configurations:

   Field	Value
   Application Name	Short
   Homepage URL	http://localhost
   Application description	URL shortening service written in Go and React
   Authorization callback URL	http://localhost/oauth/github/sign-in/callback

   

   

2. Copy Client ID and Client Secret.

   

3. Replace the value of GITHUB_CLIENT_ID in the backend/.env file with Client ID.

4. Replace the value of GITHUB_CLIENT_SECRET in the backend/.env file with Client Secret.

Create Facebook Application

1. Create a new app at Facebook Developers with the following configurations:

   Field	Value
   Display Name	Short Test
   Contact Email	your_email

2. Add Facebook Login to the app.

   

3. Copy App ID and App Secret on Settings > Basic tab.

   

4. Replace the value of FACEBOOK_CLIENT_ID in backend/.env file with App ID.

5. Replace the value of FACEBOOK_CLIENT_SECRET in backend/.env file with App Secret.

Create Google OAuth Client ID

Create a new Client ID at Google API Credentials:

1. Click on Create Credentials and select OAuth client ID.

   

2. Select Web application for Application type.

3. Fill in http://localhost/oauth/google/sign-in/callback for Authorized redirect URIs and click on Create

   

4. Replace the value of GOOGLE_CLIENT_ID in backend/.env file with Your Client ID.

5. Replace the value of GOOGLE_CLIENT_SECRET in backend/.env file with Your Client Secret.

Backend

1. Copy backend/.env.dist file to backend/.env:

   cp backend/.env.dist backend/.env

2. Update DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME, RECAPTCHA_SECRET, GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET, FACEBOOK_CLIENT_ID, FACEBOOK_CLIENT_SECRET, FACEBOOK_REDIRECT_URI, JWT_SECRET, with your own configurations.

3. Launch backend server

   cd backend
   ./scripts/dev

4. Remember to install developers tools before start coding:

   ./scripts/tools

Frontend

Remember to update REACT_APP_RECAPTCHA_SITE_KEY in frontend/.env.development.

1. Launch frontend server

   cd frontend
   ./scripts/dev

2. Visit http://localhost:3000

System Design

App Level Architecture

Short backend is built on top of Uncle Bob's Clean Architecture, the central objective of which is separation of concerns.

It enables the developers to modify a single component of the system at a time while leaving the rest unchanged. This minizes the amount of changes have to be made in order to support new requirements as the system grows. Clean Architecture also improves the testability of system, which in turn saves precious time when creating automated tests.

Here is an exmample of finance app using clean architecture:

Service Level Archtecture

Short adopts Microservices Architecture to organize dependent services around business capabilities and to enable independent deployment of each service.

Object Oriented Design

Short leverages class design, package cohesion, and package coupling princiapls to manage logical dependency between internal components.

Class Design

Principal	Description
Single Responsibility Principle	A class should have one, and only one, reason to change.
Open Closed Principle	You should be able to extend a classes behavior, without modifying it.
Liskov Substitution Principle	Derived classes must be substitutable for their base classes.
Interface Segregation Principle	Make fine grained interfaces that are client specific.
Dependency Inversion Principle	Depend on abstractions, not on concretions.

Package Cohesion

Principal	Description
Release Reuse Equivalency Principle	The granule of reuse is the granule of release.
The Common Closure Principle	Classes that change together are packaged together.
The Common Reuse Principle	Classes that are used together are packaged together.

Package Coupling

Principal	Description
Acyclic Dependencies Principle	The dependency graph of packages must have no cycles.
Stable Dependencies Principle	Depend in the direction of stability.
Stable Abstractions Principle	Abstractness increases with stability.

Dependency Injection

Short produces flexible and loosely coupled code, by explicitly providing components with all of the dependencies they need.

type Authenticator struct {
  tokenizer          fw.CryptoTokenizer
  timer              fw.Timer
  tokenValidDuration time.Duration
}

func NewAuthenticator(
  tokenizer fw.CryptoTokenizer,
  timer fw.Timer,
  tokenValidDuration time.Duration,
) Authenticator {
  return Authenticator{
    tokenizer:          tokenizer,
    timer:              timer,
    tokenValidDuration: tokenValidDuration,
  }
}

Short also simplifies the management of the big block of order-dependent initialization code with Wire, a compile time depedency injection framework by Google.

func InjectGraphQlService(
  name string,
  sqlDB *sql.DB,
  graphqlPath provider.GraphQlPath,
  secret provider.ReCaptchaSecret,
  jwtSecret provider.JwtSecret,
  bufferSize provider.KeyGenBufferSize,
  kgsRPCConfig provider.KgsRPCConfig,
) (mdservice.Service, error) {
  wire.Build(
    wire.Bind(new(fw.GraphQlAPI), new(graphql.Short)),
    wire.Bind(new(url.Retriever), new(url.RetrieverPersist)),
    wire.Bind(new(url.Creator), new(url.CreatorPersist)),
    wire.Bind(new(repo.UserURLRelation), new(db.UserURLRelationSQL)),
    wire.Bind(new(repo.URL), new(*db.URLSql)),
    wire.Bind(new(keygen.KeyGenerator), new(keygen.Remote)),
    wire.Bind(new(service.KeyFetcher), new(kgs.RPC)),

    observabilitySet,
    authSet,

    mdservice.New,
    provider.NewGraphGophers,
    mdhttp.NewClient,
    mdrequest.NewHTTP,
    mdtimer.NewTimer,

    db.NewURLSql,
    db.NewUserURLRelationSQL,
    provider.NewRemote,
    url.NewRetrieverPersist,
    url.NewCreatorPersist,
    provider.NewKgsRPC,
    provider.NewReCaptchaService,
    requester.NewVerifier,
    graphql.NewShort,
  )
  return mdservice.Service{}, nil
}

Database Modeling

Feature Toggle

Short employs feature toggles to modify system behavior without changing code. UI components controlled by the feature toggles are created inside a centralized UIFactory in order to avoid having nested if else statement across the code base:

// UIFactory.tsx
export class UIFactory {
  constructor(
    private featureDecisionService: IFeatureDecisionService
  ) {}

  public createGoogleSignInButton(): ReactElement {
    if (!this.featureDecisionService.includeGoogleSignButton()) {
      return <div />;
    }
    return (
      <GoogleSignInButton
        googleSignInLink={this.authService.googleSignInLink()}
      />
    );
  }

  public createGithubSignInButton(): ReactElement {
    if (!this.featureDecisionService.includeGithubSignButton()) {
      return <div />;
    }
    return (
      <GithubSignInButton
        githubSignInLink={this.authService.githubSignInLink()}
      />
    );
  }
}

Short also provides IFeatureDecisionService interface, allowing the developers to switch to dynamic feature toggle backend in the future by simply swapping the dependency injected.

// FeatureDecision.service.ts
export interface IFeatureDecisionService {
  includeGithubSignButton(): boolean;
  includeGoogleSignButton(): boolean;
  includeFacebookSignButton(): boolean;
}

// StaticConfigDecision.service.ts
import { IFeatureDecisionService } from './FeatureDecision.service';

export class StaticConfigDecisionService implements IFeatureDecisionService {
  includeGithubSignButton(): boolean {
    return false;
  }
  includeGoogleSignButton(): boolean {
    return false;
  }
  includeFacebookSignButton(): boolean {
    return true;
  }
}

// dep.ts
export function initUIFactory(
  ...
): UIFactory {
  ...
  const staticConfigDecision = new StaticConfigDecisionService();
  ...
  return new UIFactory(
    ...,
    staticConfigDecision
  );
}

You can read about the detailed feature toggle design on this article.

Search Engine Optimization

In order to improve the quality and quantity of the website's traffic, Short increases its visibility to web search engines through HTML meta tags.

<!-- ./frontend/public/index.html -->
<title>Short: Free online link shortening service</title>

<!-- Search Engine Optimization -->
<meta name="description"
      content="Short enables people to type less for their favorite web sites">
<meta name="robots" content="index, follow">
<link href="https://short-d.com" rel="canonical">

If you search short-d.com on Google, you should see Short shows up as the first result:

Social Media Summary Card

Facebook & LinkedIn

Short leverages Open Graph tags to control what content shows up in the summary card when the website is shared on Facebook or LinkedIn:

<!-- ./frontend/public/index.html -->
<!-- Open Graph -->
<meta property="og:title" content="Short: Free link shortening service"/>
<meta property="og:description"
      content="Short enables people to type less for their favorite web sites"/>
<meta property="og:image"
      content="https://short-d.com/promo/small-tile.png"/>
<meta property="og:url" content="https://short-d.com"/>
<meta property="og:type" content="website"/>

Shared on Facebook:

Shared on LinkedIn:

Twitter

Twitter uses its own meta tags to determine what will show up when the website is mentioned in a Tweet:

<!-- Twitter -->
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:site" content="@byliuyang11"/>
<meta name="twitter:title" content="Short: Free link shortening service"/>
<meta name="twitter:description"
      content="Short enables people to type less for their favorite web sites"/>
<meta name="twitter:image" content="https://short-d.com/promo/twitter-card.png"/>

Testing

The Importance Of Automation

Short is maintained by a small team of talented software engineers working at Google, Uber, and Vmware as a side project. The team wants to deliver new features faster without sacrificing its quality. Testing ever-increasing amount of features manually soon becomes impossible — unless we want to spend all our time with manual, repetitive work instead of delivering working features.

Test automation is the only way forward.

Testing Strategy

Please read Testing Strategies in a Microservice Architecture for a detailed introduction on test strategies.

Unit Testing

A unit test exercises the smallest piece of testable software in the application to determine whether it behaves as expected.

Run unit tests for backend:

cd backend
./scripts/unit-test

Sociable And Solitary

The FIRST Principal

• [F]ast: Unit tests should be fast otherwise they will slow down development & deployment.
• [I]ndependent: Never ever write tests which depend on other test cases.
• [R]epeatable: A repeatable test is one that produces the same results each time you run it.
• [S]elf-validating: There must be no manual interpretation of the results.
• [T]imely/[T]horoughly: Unit tests must be included for every pull request of a new feature and cover edge cases, errors, and bad inputs.

Test Structure

A automated test method should be composed of 3As: Arrange, Act, and Assert.

• [A]rrange: All the data needed for a test should be arranged as part of the test. The data used in a test should not depend on the environment in which the test is running.
• [A]ct: Invoke the actual method under test.
• [A]ssert: A test method should test for a single logical outcome.

Integration Testing

An integration test verifies the communication paths and interactions between components to detect interface defects.

Run integration tests for backend:

cd backend
./scripts/integration-test

Component Testing

A component test limits the scope of the exercised software to a portion of the system under test, manipulating the system through internal code interfaces and using test doubles to isolate the code under test from other components.

In Process

Out Of Process

Contract Testing

An integration contract test is a test at the boundary of an external service verifying that it meets the contract expected by a consuming service.

End To End Testing

An end-to-end test verifies that a system meets external requirements and achieves its goals, testing the entire system, from end to end.

The Test Pyramid

Deployment

Kubernetes

Short leverages Kubernetes to automate deployment, scaling, and management of containerized microservices.

Short uses GitOps to manage Kubernetes cluster.

Staging

Merging pull request into master branch on Github will automatically deploy the changes to staging environment.

Production

Merging from master branch to production branch will automatically deploy the latest code to the production. This is called continuous delivery in the DevOps world.

In the future, when there are enough automated tests, we may migrate to continuous deployment instead.

Tools We Use

• Drone: Continuous integration written in Go
• Code Climate: Automated code review
• ElephantSQL: Managed PostgreSQL service.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, the process for submitting pull requests to us, and our code review guideline.

Author

Harry Liu - Initial work - byliuyang

As the tech lead of Short, I am responsible for the overall planning, execution and success of complex software solutions to meet users' needs.

I deeply believe in and am striving to achieve the right column of the following diagram:

License

This project is maintained under MIT license