11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

(wip)MFV trails (AA)

Published at March 11, 2021, 5:16 PM Edited at September 1, 2021, 4:55 PM

Copied from: CEO Office/Service Organization Control (SOC)/MFJ trails

Before going...
Representative / PIC
Managing by this repository
Release management ledger (SOC対象リリース管理台帳)
QualityPIC management ledger (品質管理担当者管理台帳)

1. Application/IT infrastructure management

1.1 manage Standards
Move the MFJ one to Kibela & Translate Standards in English
Trail: the list of InformationSecurityRepresentative
Same as MFJ (the members who manage Standards, i.e. MFJ CISO office members)

1.2 begin developments/changes

Infra
Assign QualityPIC
https://github.com/moneyforwardvietnam/mfv-
infra/blob/master/.github/CODEOWNERS
Trail: PR screen Starting development/chagnes infrastructure（開発・変更案件の着手_イ
ンフラ）
https://moneyforward.kibe.la/notes/186711#infra-3 1/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Trail: PR notification Starting development/changes infrastructure（開発・変更案件の着手

_インフラ）

Dev
Assign QualityPIC
Make reviewers team on Github <- Add QualityPIC
https://github.com/orgs/moneyforwardvietnam/teams/asset_accounting_reviewe
rs
Add Github setting to require at least 1 approval by QualityPIC to merge
https://moneyforward.kibe.la/notes/186711#infra-3 2/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

asset_accounting_backend

asset_accounting_frontend

Trail: PR screen Starting development/changes （PRの起案画面）

Trail: PR notification Starting development/changes （PRの起案通知）

https://moneyforward.kibe.la/notes/186711#infra-3 3/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

1.3 test developments/changes

Infra
Add "merge blocker" unless CI succeeds
Trail: CI hook (integration) setting / CI on PR setting

Trail: Succeed on CI

Dev
Add "merge blocker" unless CI succeeds
Trail: Integrate CI
1 CI hook setting
asset_accounting_backend
https://moneyforward.kibe.la/notes/186711#infra-3 4/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

asset_accounting_frontend

2 CI on PR setting
asset_accounting_backend

https://moneyforward.kibe.la/notes/186711#infra-3 5/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

asset_accounting_frontend

Trail: Succeed on CI
asset_accounting_backend

https://moneyforward.kibe.la/notes/186711#infra-3 6/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

asset_accounting_frontend

1.4 release developments/changes on Prod

Infra
Trail: Unable to approve your own PR
PR screen

https://moneyforward.kibe.la/notes/186711#infra-3 7/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Dev
Trail: Unable to approve your own PR
Grayout screen (disabled button)
asset_accounting_backend

asset_accounting_frontend

Trail: Write permission setting screen (able to create PR by ChangePIC )

https://moneyforward.kibe.la/notes/186711#infra-3 8/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

asset_accounting_backend

asset_accounting_frontend

Trail: Slack notification of deployment on prod (Need ChangeRepresentative in the

channel)
asset_accounting_backend

asset_accounting_frontend

https://moneyforward.kibe.la/notes/186711#infra-3 9/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

1.5 manage developments/changes

Trail: List up all PRs to master / main branch
https://docs.google.com/spreadsheets/d/17FJyO72lJD2fA-
I1gKySwtP2SfY15RfVBvxxQkzeX5w

1.6 manage structures

Trail: Application directory structure
asset_accounting_backend

https://moneyforward.kibe.la/notes/186711#infra-3 10/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

asset_accounting_frontend

Trail: Schemafile of Database

https://moneyforward.kibe.la/notes/186711#infra-3 11/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Trail: .circleci/config.yml to prove deployment by CI

asset_accounting_backend

asset_accounting_frontend

https://moneyforward.kibe.la/notes/186711#infra-3 12/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

2. Data patch management

2.1 manage standards
same as 1.1

2.2 Approval to changes, tests and service envrironments

trail: Issue to apply data patch

https://moneyforward.kibe.la/notes/186711#infra-3 13/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

trail: workflow for the data patch

trail: data patch management ledger

Items in data patch ledger

name of implement , approve , execute and confirm can be English name,
GitHubID and real name.
item of columns: can be minumum at first

https://moneyforward.kibe.la/notes/186711#infra-3 14/24
 2.3 Operation to execute on service environments
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

No need. Same flow as MFJ, Cybozu and access list

2.4 Separation of responsibilities

No need. PWS by MF

3. Operation management (SRE)

3.1 manage standards
same as 1.1

3.2 register Jobs, manage changes

No need to care because we'll use application codes to change
If developpers need to change manually, need a Trail

3.3 monitor applications, infrastractures

Infra
Trail: DB Backup job settin

https://moneyforward.kibe.la/notes/186711#infra-3 15/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Trail: Slack notification of DataDog login checker <= add developers in the channel

Trail: (PagerDuty) setting page to call the product members

Trail: APM metrics alert to Slack
APM notification is necessary in the point of SOC?

3.4 manage monitoring targets

Trail: (External Monitoring) Configure by code

https://moneyforward.kibe.la/notes/186711#infra-3 16/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Trail: (External Monitoring) Configuration code PR screen

Approval by QualityPIC

Trail: (External Monitoring) Slack notification of PR

Trail: (External Monitoring) Terraform Cloud to apply above things

TODO

4. System failures management (SRE)

4.1
https://moneyforward.kibe.la/notes/186711#infra-3 17/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

same as 1.1

4.2 manage system failures

Create #incident_tracking channel
We created the #incident_tracking_vn in Slack

The configuration DataDog to this channel

https://moneyforward.kibe.la/notes/186711#infra-3 18/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Trail: (PagerDuty) history page

Trail: Slack notification of DataDog alert
Trail: SLA management sheet
write approval comment in report
https://moneyforward.kibe.la/notes/186711#infra-3 19/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Make System failure report template

none
Trail: System failure report
use the above template
Trail: System failure sharing meeting log
none

5. Backup management (SRE)

5.1 Backup application and data
Trail: DB backup log

Trail: DB backup configuraton or code (script if exist)

Code not exist because it's aurora's default setting.
DB automated backups

https://moneyforward.kibe.la/notes/186711#infra-3 20/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

5.2 Execute recovery test

Trail: Recovery test procedure on esa or github
Please following the procedure in ESA for details
Trail: Recovery test procedure log
AWS Management Console and open the Amazon RDS console

choose Snapshots. -> Choose the DB cluster snapshot that we want to restore from
-> For Actions, choose Restore snapshot.

https://moneyforward.kibe.la/notes/186711#infra-3 21/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

On the Restore snapshot page, for DB instance identifier, enter the name for our
restored DB cluster.

Choose Restore DB instance -> The DB instance is creating \

DB instances is available

https://moneyforward.kibe.la/notes/186711#infra-3 22/24
 6. Access management
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

6.1 manage standards

same as 1.1

6.2 restrict access from outside (application layer access

management)
Restiriction on Guwakamole (MFJ)

6.3 manage access to service environments

CybozuWF to grant permissions to PWS

6.4 manage Master account

MFJ handles it

6.5 manage password

MFJ handles it

6.6 create/revoke accounts (申請からのアカウント追加、退職

情報からのアカウント削除)
PWS
(Current) mf.moneyforward.vn --> Adure AD --> PWS
Need to apply everytime a member retires ref
Use cybozu to add and delete PWS accounts
(Within this year) MFV Google workspace --> Adure AD --> PWS
When MFV GSuite accout is revoked, he or she cannot login to PWS`
Other accounts
https://moneyforward.kibe.la/notes/186711#infra-3 23/24
11/4/21, 3:41 PM (wip)MFV trails (AA) | moneyforward Kibela

Accounts management target list

off-boarding
MFV corp infra tells MFJ team to revoke PWS accounts

6.7 renew accounts (MFPWS アカウント棚卸)

Bamboo will join the channel and review the accounts quarterly

6.8 get and check logs

PWS by MFJ

https://moneyforward.kibe.la/notes/186711#infra-3 24/24