Professional Documents
Culture Documents
V2 Holyjs 2019 - Nestjs. Tried To Shift in 80 Hours
V2 Holyjs 2019 - Nestjs. Tried To Shift in 80 Hours
V2 Holyjs 2019 - Nestjs. Tried To Shift in 80 Hours
Tried To Shift
In 80 Hours
1
2
3
4
Tools
5
Tools
● npm packages
6
package.json
"dependencies": {
"bcrypt-nodejs": "0.0.3",
"crowdin": "1.0.0",
"express-tgz": "0.0.3",
"gitignore-to-glob": "0.3.0",
"migration": "0.3.0",
"nconf": "0.8.4",
"node-uuid": "1.4.8",
"shorturl-2": "0.0.6",
"unzip": "0.1.11"
}
7
package.json
"dependencies": {
"bcrypt-nodejs": "0.0.3",
"crowdin": "1.0.0",
"express-tgz": "0.0.3",
"gitignore-to-glob": "0.3.0",
"migration": "0.3.0",
"nconf": "0.8.4",
"node-uuid": "1.4.8",
"shorturl-2": "0.0.6",
"unzip": "0.1.11"
}
8
package.json
"dependencies": {
"bcrypt-nodejs": "0.0.3",
"crowdin": "1.0.0",
"express-tgz": "0.0.3",
"gitignore-to-glob": "0.3.0",
"migration": "0.3.0",
"nconf": "0.8.4",
"node-uuid": "1.4.8",
"shorturl-2": "0.0.6",
"unzip": "0.1.11"
}
9
package.json
"dependencies": {
"bcrypt-nodejs": "0.0.3",
"crowdin": "1.0.0",
"express-tgz": "0.0.3",
"gitignore-to-glob": "0.3.0",
"migration": "0.3.0",
"nconf": "0.8.4",
"node-uuid": "1.4.8",
"shorturl-2": "0.0.6",
"unzip": "0.1.11"
}
10
Tools
● npm packages
● express
11
Express 1
app.use(session({
secret: 'keyboard cat',
proxy: true,
resave: true,
saveUninitialized: true
}));
12
Express 2
module.exports = () => {
AWS.config.region = conf.S3_REGION;
AWS.config.update({
accessKeyId: conf.S3_ACCESS_KEY_ID,
secretAccessKey: conf.S3_SECRET_ACCESS_KEY
});
AWS.config.region = conf.S3_REGION;
AWS.config.update({
accessKeyId: conf.S3_ACCESS_KEY_ID,
secretAccessKey: conf.S3_SECRET_ACCESS_KEY
});
16
Database 1: Issues
XXXX-XX-XXTXX:XX:XX.XXX+0000 I NETWORK [thread1] connection
accepted from xxx.xxx.xxx.xxx:41681 #27019 (1611 connections now open)
XXXX-XX-XXTXX:XX:XX.XXX+0000 F - [statsSnapshot] out of memory.
17
Database 2: No Structure
[{
"show": "true",
"isTrash": false
}, {
"show": false,
"hidden": "hide"
}, {
"hidden": true
}, {
"list": "white",
"hidden": "show"
}]
18
Database 3: Legacy props
const shouldBeShown =
thingHidden === LEGACY_VALID_SHOW_VALUE
|| thingHidden !== LEGACY_VALID_HIDDEN_VALUE;
19
20
What did this go to lead to?
21
High-level Defects
● Unpredictable break down
22
High-level Defects
● Unpredictable break down
● Available free-paid content
23
High-level Defects
● Unpredictable break down
● Available free-paid content
● Loss of business
24
High-level Defects
● Unpredictable break down
● Available free-paid content
● Loss of business
● Hydra Bug
25
A Vital Question
26
Solutions
27
Solutions
● add new middleware
28
Solutions
● add new middleware
● make data refactoring
29
Solutions
● add new middleware
● make data refactoring
● increase cyclomatic complexity
30
Solutions
● add new middleware
● make data refactoring
● increase cyclomatic complexity
● call for psychics || resignation notice
31
Wrapper?
32
Nest is a platform-agnostic framework.
[...]
For example, most components can be re-used
without change across different underlying HTTP
server frameworks (e.g., Express and Fastify).
33
Express Nest
● custom modules & architecture ✓ tuned up & re-usable logic
● too many efforts for ✓ DI
reorganizing ✓ AOP (e.g. Interceptors)
● writing tests too complex ✓ Angular
● potential vulnerabilities
34
35
I'll take
your
wager!
36
Worthwhile cause
37
Worthwhile cause
● accident prevention and response
● discovery hidden defects
● detect legacy data
38
Worthwhile cause
● accident prevention and response
● discovery hidden defects
● detect legacy data
● minimal changes
39
40
80:00
41
Steps
● Quantify the amount of work
42
Repository
43
● Monorepo
44
● Monorepo
● 3 subproject
45
● Monorepo
● CMS javascript node v8.12.0
46
● Monorepo
● CMS javascript node v8.12.0
● CRON javascript node v8.12.0
47
● Monorepo
● CMS javascript node v8.12.0
● CRON javascript node v8.12.0
● API typescript node v2.7.2
48
● Monorepo
● CMS javascript node v8.12.0
● CRON javascript node v8.12.0
● API typescript node v2.7.2
● 3 common modules
49
● Monorepo
● CMS javascript node v8.12.0
● CRON javascript node v8.12.0
● API typescript node v2.7.2
● 3 common modules
● 0 Business Logic Layers
● e2e tests
50
jscpd
51
API CMS
53
Steps
✓ Quantify the amount of work
54
55
78:00
56
Steps
✓ Quantify the amount of work
● Create Nestjs App
57
Add core packages
➔ npm i \
@nestjs/common
58
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core
59
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0
60
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0
61
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0 \
typescript
62
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0 \
typescript \
reflect-metadata
63
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0 \
typescript \
reflect-metadata \
@nestjs/platform-express
64
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0 \
typescript \
reflect-metadata \
@nestjs/platform-express
https://github.com/nestjs/nest/issues/1609
65
Add core packages
➔ npm i \
@nestjs/common \
@nestjs/core \
rxjs@6.0.0 \
typescript \
reflect-metadata \
@nestjs/platform-express
66
Add dev packages
➔ npm i -D @types/node \
ts-node
67
tsconfig.json -> compilerOptions
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
68
Create Nest app
import { NestFactory } from '@nestjs/core';
import { ApplicationModule } from './application.module';
bootstrap();
69
Create Application module
import { Module } from '@nestjs/common';
@Module({
imports: [],
controllers: []
})
export class ApplicationModule {
}
70
Express server
71
Before After
app.listen(port, () => {
console.log(`CMS
module.exports.cms = app;
listening on ${port}`);
});
72
Before After
import { NestFactory } from '@nestjs/ import { NestFactory } from '@nestjs/
core'; core';
import { ApplicationModule } from './ import { ApplicationModule } from './
app.module'; app.module';
import { ExpressAdapter } from
'@nestjs/platform-express';
import { cms } from
'./cms';
73
Before After
74
➔ curl "http://localhost:8081/v1/login" -X POST
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /</pre>
</body>
</html>
75
passport.use('local', new LocalStrategy(
(email, password, done) => ...
));
76
➔ curl "http://localhost:8081/v1/login?email=…&password=…" -X GET
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /</pre>
</body>
</html>
77
'use strict';
const initialize = require('./init');
78
return new Promise(function (resolve, reject) {
async.parallel({
a: Entity.findOne({name: 'Value A' }),
b: Entity.findOne({name: 'Value B' }),
c: Entity.findOne({name: 'Value C’ })
}, (error, result) =>
error ? null : resolve(result)
);
});
79
➔ curl "http://localhost:8081/v1/login?email=…&password=…" -X GET
80
Express server 2
81
import { api } from ‘./api';
await appCms.listen(8081);
await appApi.listen(8082);
82
Cron
83
Add package
➔ npm i nest-schedule
84
Module
import { Module } from '@nestjs/common';
import { ScheduleModule } from 'nest-schedule';
import { ScheduleService } from './schedule.service';
@Module({
imports: [
ScheduleModule.register({}),
],
providers: [ScheduleService],
})
export class CronModule {}
85
Service
import { Injectable } from '@nestjs/common';
import { Interval, NestSchedule } from 'nest-schedule';
@Injectable()
export class ScheduleService extends NestSchedule {
@Interval(2000)
intervalJob() {
console.log('executing interval job');
return false;
}
}
86
const appCms = await NestFactory.create(
ApplicationModule,
new ExpressAdapter(cms)
);
const appApi = await NestFactory.create(
ApplicationModule,
new ExpressAdapter(api)
);
const appCron = await NestFactory.createApplicationContext(CronModule);
await appCms.listen(8081);
await appApi.listen(8082);
await appCron.init();
87
➔ npm start
[Nest] - [NestFactory] Starting Nest application...
[Nest] - [InstanceLoader] AppModule dependencies initialized +12ms
[Nest] - [NestFactory] Starting Nest application... +2ms
[Nest] - [InstanceLoader] AppModule dependencies initialized +3ms
[Nest] - [NestFactory] Starting Nest application... +1ms
[Nest] - [InstanceLoader] CronModule dependencies initialized +4ms
[Nest] - [InstanceLoader] ScheduleModule dependencies initialized +0ms
[Nest] - [RoutesResolver] AppController {/}: +5ms
[Nest] - [RouterExplorer] Mapped {/, GET} route +3ms
[Nest] - [NestApplication] Nest application successfully started +1ms
[Nest] - [RoutesResolver] AppController {/}: +7ms
[Nest] - [RouterExplorer] Mapped {/, GET} route +2ms
[Nest] - [NestApplication] Nest application successfully started +1ms
executing interval job
executing interval job
executing interval job
88
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
89
90
72:00
91
Migrate to typescript?
92
Migrate from js to ts?
● 3 common modules between javascript & typescript apps
93
Migrate from js to ts?
● 3 common modules between javascript & typescript apps
● static type checking
94
Migrate from js to ts?
● 3 common modules between javascript & typescript apps
● static type checking
● typescript or Babel for nodejs v8.12.0 (>= 8.9.0)
95
Migrate from js to ts?
● 3 common modules between javascript & typescript apps
● static type checking
● typescript or Babel for nodejs v8.12.0 (>= 8.9.0)
● nestjs fully supports typescript
96
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
● Migrate from js to ts
97
Migration
98
js -> ts
'use strict';
const initialize = require('./init');
'use strict';
const initialize = import { initialize } from
require('./init'); ‘./init';
100
Before After
101
Before After
102
Before After
import { authorization } from './auth;
initialize(app)
initialize(app)
.then(() => { .then(() => {
require('./auth')(app); authorization(app);
}) })
.catch((error) => .catch((error) =>
console.log(error) console.log(error)
); );
103
javascript
'use strict';
const initialize = require('./init');
106
● jscodeshift + extensions
107
● jscodeshift + extensions
108
● jscodeshift + extensions https://github.com/cpojer/js-codemod
#jscodeshift-imports
#template-literals
#arrow-function
#rm-requires
#no-vars
109
● jscodeshift + extensions
● IDE (webstorm)
110
● jscodeshift + extensions
● IDE (webstorm)
111
● jscodeshift + extensions
● IDE (webstorm)
● linter --fix
112
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
113
114
50:00
115
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
● Nestjs simple modules
116
Serve Static
@Module
117
➔ npm i @nestjs/serve-static
118
Before After
119
Nest app
import { NestFactory } from '@nestjs/core';
import { ApplicationModule } from './app.module';
bootstrap();
120
CORS
@Module
121
Nest app
import { NestFactory } from '@nestjs/core';
import { ApplicationModule } from './app.module';
bootstrap();
122
Load .env
@Provider
123
Before After
124
Before After
https://docs.nestjs.com/techniques/configuration
125
DB Connection
@Module
126
➔ npm install --save @nestjs/mongoose mongoose
127
Before After
export const dbConfig = (…) => { @Module({
const opt = { imports: [
useNewUrlParser: true, …
connectTimeoutMS: 5000
MongooseModule.forRoot(…)
};
]
mongoose.connect(…, err => })
if (err) console.error(err);
);
};
128
Before After
export const dbConfig = (…) => { @Module({
const opt = { imports: [
useNewUrlParser: true, …
connectTimeoutMS: 5000
MongooseModule.forRoot(…)
};
]
mongoose.connect(…, err => })
if (err) console.error(err);
);
};
https://docs.nestjs.com/techniques/mongodb#async-configuration
129
Mongoose Schemas
130
Before After
const schema = new
mongoose.Schema({
…
});
mongoose.model('User', schema);
131
Backward compatibility
132
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
133
134
46:00
135
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
● Nestjs Auth module
136
Passport
@Module
137
➔ npm install --save @nestjs/passport passport
138
Before After
app.use(passport.initialize()); @Module({
app.use(passport.session()); imports: [
…
PassportModule.register({
session: true
})
]
})
139
Local Strategy
@CustomProvider
140
➔ npm install --save passport-local
➔ npm install --save-dev @types/passport-local
141
Before After
142
Before After
144
Before After
isAdmin: (req, res, next) => { import { SetMetadata } from '@nestjs/
if ( common';
req.user &&
req.user.role === 'admin'
export const Roles =
)
(...roles: string[]) =>
return next();
SetMetadata('roles', roles);
res.redirect('/');
}
145
export class RolesGuard implements CanActivate {
canActivate(context: ExecutionContext): boolean {
}
}
146
Before After
const isAdmin = (req, res, next) => @Module({
req.user.role === 'admin' providers: [
? next() {
: res.redirect('/'); provide: APP_GUARD,
useClass: RolesGuard,
app.get(`/api/user/:id`, isAdmin, getUser()); }
]
})
147
Before After
const isAdmin = (req, res, next) => @Module({
req.user.role === 'admin' providers: [
? next() {
: res.redirect('/'); provide: APP_GUARD,
useClass: RolesGuard,
app.get(`/api/user/:id`, isAdmin, getUser()); }
]
})
https://github.com/marcomelilli/nestjs-email-authentication
148
To do
149
● Compression @Middleware ● Upload images @Interceptor
● Caching @Interceptor ● Logging @Middleware
● Validation @Middleware @Pipe ● Websocket
● CSRF @Middleware ● Health check
● Rate limiting @Middleware
150
https://docs.nestjs.com/middleware
https://docs.nestjs.com/pipes
https://docs.nestjs.com/interceptors
151
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
152
153
42:00
154
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
● Refactoring Controllers
155
Controller
156
Before After
const isAdmin =
(req, res, next) =>
req.user.role === 'admin'
? next()
: res.redirect('/');
157
Before After
app.setGlobalPrefix('v1');
@Controller('/user')
158
Before After
app.put(..., async editUser
function editUser (req, res) {...} (@Body() userDto: userDto) {
return
);
this.service.editUser(userDto);
}
async getUser
app.get(..., getUser()); (@Params() id: ObjectId) {
function getUser() { return
return (req, res) => {...} this.service.getUser(id);
} }
159
Controller Before
const isAdmin = (req, res, next) =>
req.user.role === 'admin' ? next() : res.redirect('/');
function getUser() {
return (req, res) =>
Users.findOne(...).exec(() => res.json({...}))
}
160
Controller After
import { Body, Controller, Get, Param, Put } from '@nestjs/common';
import { EditUserDto, UsersService } from './user.service';
import { Roles } from './roles.decorator';
@Controller('/user')
export class AppController {
constructor(private readonly userService: UsersService) {}
@Put(`/:id`)
@Roles('admin')
async editUser (@Body() editUserDto: EditUserDto) {
return this.userService.editUser(editUserDto);
}
}
161
Service
162
Before After
app.get(..., getUser());
163
Before After
app.put(...,
function editUser (req, res) { editUser (userDto: UserDto) {
Users.update(...).exec( this.users
() => res.json({...}) .update(...).exec(...);
); }
}
);
164
Before After
app.put(..., async
function editUser (req, res) { editUser (userDto: UserDto) {
Users.update(...).exec( this.users
() => res.json({...}) .update(...).exec(...);
); }
}
);
165
Before After
app.put(..., async
function editUser (req, res) { editUser (userDto: UserDto) {
Users.update(...).exec( return this.users
() => res.json({...}) .update(...).exec(...);
); }
}
);
166
Before After
app.put(..., async
function editUser (req, res) { editUser (userDto: UserDto) {
Users.update(...).exec( return this.users
() => res.json({...}) .update(...).exec();
); }
}
);
167
Service Before
const isAdmin = (req, res, next) =>
req.user.role === 'admin' ? next() : res.redirect('/');
function getUser() {
return (req, res) =>
Users.findOne(...).exec(() => res.json({...}))
}
168
Service After
import { Injectable } from '@nestjs/common';
import { InjectModel } from '@nestjs/mongoose';
import { Model } from 'mongoose';
@Injectable()
export class UserService {
constructor(@InjectModel('User') users: Model<User>) {}) {}
170
function transformer(fileInfo, api, options) {
const j = api.jscodeshift;
const root = j(fileInfo.source);
root.find(j.ClassDeclaration)
.replaceWith(p => {
p.node.decorators = [
j.decorator(j.callExpression(j.identifier('Injectable'), []))
];
return p.node;
});
return root.toSource();
};
171
e2e tests
172
➔ npm run e2e
I/testLogger -
I/launcher - 0 instance(s) of WebDriver still running
✓ I/launcher - chrome #01-0 passed ✓ I/launcher - chrome #01-2 passed
✓ I/launcher - chrome #01-1 passed ✓ I/launcher - chrome #01-12 passed
✓ I/launcher - chrome #01-3 passed ✓ I/launcher - chrome #01-15 passed
✓ I/launcher - chrome #01-4 passed ✓ I/launcher - chrome #01-13 passed
✓ I/launcher - chrome #01-6 passed ✓ I/launcher - chrome #01-16 passed
✓ I/launcher - chrome #01-5 passed ✓ I/launcher - chrome #01-17 passed
✓ I/launcher - chrome #01-9 passed ✓ I/launcher - chrome #01-14 passed
✓ I/launcher - chrome #01-10 passed ✓ I/launcher - chrome #01-18 passed
✓ I/launcher - chrome #01-8 passed ✓ I/launcher - chrome #01-7 passed
✓ I/launcher - chrome #01-11 passed
173
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
174
175
08:00
176
177
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
● Performance testing
178
Add dev packages
➔ npm i -D \
artillery
artillery-plugin-expect
179
Before After
Scenarios launched: 300 Scenarios launched: 300
Scenarios completed: 300 Scenarios completed: 300
RPS sent: 9.95 RPS sent: 9.87
Request latency: Request latency:
min: 140.4 min: 150.2
max: 1096.4 max: 1295.8
median: 365.2 median: 386.7
p95: 601.3 p95: 769.1
p99: 755.3 p99: 915.2
180
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
✓ Performance testing
181
182
04:00
183
Worthwhile cause
● accident prevention and response
● discovery hidden defects
● detect legacy data
● minimal changes
184
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
✓ Performance testing
● Business value
185
@Interceptor
186
● bind extra logic before / after method execution
● transform the result returned from a function
● transform the exception thrown from a function
● extend the basic function behavior
● completely override a function depending on specific
conditions (e.g., for caching purposes)
187
● bind extra logic before / after method execution
● transform the result returned from a function
● transform the exception thrown from a function
● extend the basic function behavior
● completely override a function depending on specific
conditions (e.g., for caching purposes)
188
Response
@Interceptor
189
export class ResponseInterceptor {
intercept(context, next) {
return next.handle().pipe( tap( data =>
const res = context.switchToHttp().getResponse();
if (!isCustomResponse(data)) {
logger.log(...);
}
return res.json(data);
190
Timeout
@Interceptor
191
export class TimeoutInterceptor {
intercept(context, next) {
return next.handle().pipe( timeout( 5000 ) )
}
}
192
Controller
193
@Controller()
@UseInterceptors(TimeoutInterceptor, ResponseInterceptor)
export class AppController {}
194
Steps
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
✓ Performance testing
✓ Business value
195
Conclusion
00:00
196
197
Checklist
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
✓ Performance testing
✓ Business value
198
Checklist
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
✓ Performance testing
✓ Business value
199
Checklist
✓ Quantify the amount of work
✓ Create Nestjs App
✓ Performance testing
✓ Migrate from js to ts
✓ Nestjs simple modules
✓ Nestjs Auth module
✓ Refactoring Controllers
✓ Performance testing
✓ Business value
200
Todo Nest
201
Todo Repo
● Separate common DB layer (repository)
202
Todo Repo
● Separate common DB layer (repository)
● Health check
203
Todo Repo
● Separate common DB layer (repository)
● Health check
● Refactoring
204
Todo Repo
● Separate common DB layer (repository)
● Health check
● Refactoring
● Tests, tests, tests
205
HOLYJS, MOSCOW 2019
● https://github.com/nestjs/nest/issues/1609
● https://github.com/facebook/jscodeshift
● https://github.com/cpojer/js-codemod
● https://docs.nestjs.com/techniques/configuration
● https://docs.nestjs.com/techniques/mongodb#async-configuration
● https://docs.nestjs.com/techniques/authentication
● https://docs.nestjs.com/guards#putting-it-all-together
● https://github.com/marcomelilli/nestjs-email-authentication
● https://medium.com/@kyuwoo.choi/sneak-peek-to-javascript-
aop-16458f807842
206
HOLYJS, MOSCOW 2019
?
207