Architecture and Infrastructure
Cache-Aside
- Advantages
- Resilient to cache failures
- The data model in the cache may differ from the database model
- Disadvantages
- Slow data updates
- Data in the cache can become inconsistent with the database
Read-Through
- Advantages
- Less application complexity and a lower chance of errors
- Disadvantages
- Cache miss on the first request
- Limited choice of the cache data model
- More sensitive to errors than Cache-Aside
Refresh-Ahead
- Advantages
- Low cost of reading data from the database
- Consistency of frequently accessed cache entries
- Disadvantages
- The cache must work flawlessly: errors are not detected immediately and lead to data inconsistency and reads from the database
Write-Through
- Advantages
- Data between the cache and the database is always synchronized
- Disadvantages
- Requires waiting for the cache to update the database
Write-Behind
- Time-based
- Purge cache
- Refresh cache
- TTL (Time-To-Live) expiration
- Key-based
Availability = (Agreed availability time − Downtime) / Agreed availability time × 100
- Active-Active
- Active-Passive (Active-Standby)
- Active-Cold Standby
- MTBF — mean time between failures
- MTTR — mean time to recovery
- MTTD — mean time to detect
- Repair time
- RTO (Recovery Time Objective)
- RPO (Recovery Point Objective)
- WRT (Work Recovery Time)
- MTPD (Maximum Tolerable Period of Disruption)
- Mission critical
- Business critical
- Business operational
- Office productivity
- Backpressure
- Circuit Breaker
- Rate limiting
- Types
- Blocking
- Throttling
- Prioritization
- Algorithms
- Token bucket
- Fixed window counter
- Sliding window counter
- Sliding window log
- Types
- Bulkhead
- Transactional outbox
- Polling publisher
- Transaction log tailing
- Change Data Capture
- Testing
- D&D game
- USE
- Utilization
- Saturation
- Errors
- 4 Golden Signals
- Latency
- Traffic
- Errors
- Saturation
- RED
- Request rate
- Errors
- Duration
- Deployment models
- Private cloud
- Public cloud
- Hybrid cloud
- Service models
- IaaS — Compute, Storage, Network
- PaaS — Platform, Runtime, Services
- SaaS — Software, Infrastructure, Support
- Retain — partial migration.
- Retire — replace part of the functionality with cloud solutions.
- Rehost (Lift and Shift) — migrate without changes.
- Repurchase (Drop and Shop) — abandon the current implementation and move fully to cloud solutions.
- Replatform (Lift and Reshape) — minimal changes to benefit from cloud solutions.
- Refactor (Re-architect) — completely rework the current implementation for the cloud.
- Business requirements & technology stack definition
- Data acquisition
- Data preprocessing
- Data analytics
- Model development
- Model & solution validation
- Decision deployment
- Model monitoring
- Business understanding
- Data understanding
- Data preparation
- Data modeling
- Evaluation
- Deployment
- Lost update
- Dirty read
- Non-repeatable read
- Phantom read
- ACID
- Atomicity
- Consistency
- Isolation
- Durability
- BASE
- Basically available
- Soft state
- Eventual consistency
- Read uncommitted
- Read committed
- Repeatable read
- Serializable
| Isolation level | Lost update | Dirty read | Non-repeatable read | Phantom read |
|---|---|---|---|---|
| Read uncommitted | No | Yes | Yes | Yes |
| Read committed | No | No | Yes | Yes |
| Repeatable read | No | No | No | Yes |
| Serializable | No | No | No | No |
- Data warehouse
- Modern data warehouse
- Data lake
- Raw data store
- Data lakehouse
- Types of storage
- File storage
- Block storage
- Object storage
- Amazon Simple Storage Service (S3)
- Metadata
- Apache Iceberg — open file formats, open table formats
- Apache Hudi
- Delta Lake
- Catalogs
- Based on files
- Hadoop catalog
- Based on services
- JDBC
- Hive Metastore
- Nessie
- Glue
- Based on files
- Types of storage
- ETL (Extract, Transform, Load)
- ELT (Extract, Load, Transform)
- Bill Inmon
- Ralph Kimball
- Data Vault
- Anchor
- Star
- Snowflake
- Column-oriented
- MPP (Massive Parallel Processing)
- MapReduce
- Apache Spark
- Row-oriented
- Batch-processing engine
- Real-time processing engine
- TOGAF (Open Group Architecture Framework)
- IT landscape map/diagram
- Business capability map
- Integration diagram
- Digital Transformation Roadmap
- Project-based approach
- PRINCE2
- PMBOK
- Product-based approach
- Project-based approach
| System | Used by | Purpose |
|---|---|---|
| ERP (Enterprise Resource Planning) | CFOs, operations managers | Manage enterprise resources, integrate finance, production, and logistics |
| BPMS (Business Process Management System) | Process owners, analysts | Model, automate, and optimize business processes |
| CRM (Customer Relationship Management) | Sales, marketing teams | Manage customer relationships and enhance sales performance |
| MDM (Master Data Management) | Data governance teams | Ensure data consistency and quality across systems |
| BI (Business Intelligence) | Executives, decision-makers | Data analysis, reporting, strategic decision-making |
| PLM (Product Lifecycle Management) | Product development teams | Manage the entire product lifecycle |
| PDM (Product Data Management) | Engineering, design teams | Manage product data and documentation |
| MES (Manufacturing Execution System) | Production managers | Oversee and optimize manufacturing processes |
| SCADA (Supervisory Control and Data Acquisition) | Plant operators, engineers | Monitor and control industrial processes |
| ABS (Automated Banking System) | Bank employees, customers | Process financial transactions, manage accounts, provide banking services |
| SCM (Supply Chain Management) | Logistics, supply chain managers | Manage and optimize supply chains |
- Row — business capability
- Column — organizational unit
- Intersection — system
- Architectural debt
- Codebase debt
- Infrastructure debt
- Software quality debt
- Process debt
- Defect debt
- Documentation debt
- Intentional/known debt
- Unknown legacy/accidental debt
- Bit rot/band-aid debt
- Cost estimation
- Criticality assessment
- Technical Debt Ratio (TDR) = (cost of elimination / development cost) × 100%
- Tolerate
- Invest
- Eliminate
- Mitigate
- Quadrants
- Languages and frameworks
- Tools
- Platforms & infrastructure
- Methods & patterns
- Rings
- Base (Adopt, Trial, Assess, Hold)
- TRL (Technology Readiness Levels)
- Contents
- Basic principles observed and reported
- Technology concept formulated
- Experimental proof of concept
- Technology validated in lab environment
- Development 5. Technology validated in relevant environment 6. System/subsystem prototype demonstrated in relevant environment 7. System prototype demonstrated in operational environment 8. Actual system completed and qualified through test and demonstration
- Implementation 9. Actual system proven through successful mission operations
- Contents
- Trends
- New
- Moved in/out
- No change
- FURPS+
- Functionality
- Usability
- Reliability
- Performance
- Supportability
- Restrictions
- Use Case Flow / User Story
- Basic flow
- Alternative flow
- BPMN
- Mendelow matrix
- Business Model Canvas
- Lean Canvas
- SAFe (Scaled Agile Framework)
- Value stream
- Stakeholders
- Stakeholder matrix
- High/low engagement
- High/low influence
- Role
- Tasks
- Requirements and expectations
- Stakeholder matrix
- Predictive approach
- Reactive approach
- Sequential approach
- Architecture vision
- Architecture Decision Record (ADR)
- Y-statement
- Context
- Facing
- We decided for
- Neglected
- To achieve
- Accepting that
- Y-statement
- Architecture Decision Log
- Architecture Decision Record (ADR)
- Change management
- Design thinking
- Empathize
- Define
- Ideate
- Prototype
- Test
- Storytelling
- PII (Personally Identifiable Information)
- PHI (Protected Health Information)
- Proactive not reactive; preventative not remedial
- Privacy Impact Assessment (PIA)
- Privacy as the default setting
- Privacy embedded into design
- Full functionality — positive-sum, not zero-sum
- End-to-end security — full lifecycle protection
- Visibility and transparency
- Respect for user privacy
- Data Flow Diagram (DFD)
- Data source
- Processes
- Data stores
- Data flows
- External entities
- Digital Rights Management (DRM)
- Collective knowledge
- Raw data
- Processed data
- Golden source
- At rest
- During transfer process
- During use
- Scope definition and planning
- Data discovery and inventory
- Data flow mapping
- Data transformation documentation
- Metadata management
- Tool selection and implementation
- Regulatory compliance assurance
- Governance framework and policy establishment
- Continuous improvement and monitoring
- Data governance
- Data minimization
- Data obfuscation mechanisms
- Data masking
- Tagging data
- Team functional boundaries
- Cross-functional collaboration
- Organizational social boundaries
- Man
- Machine
- Material
- Method
- Measurement
- Mother nature
- DORA
- Deployment frequency
- Lead time for changes
- Change failure rate
- Time to restore service
- MTTR (Mean Time to Recovery)
- Error rate
- Performance metrics
- RTO (Recovery Time Objective) & RPO (Recovery Point Objective)
- Gradual replacement of the monolith
- Reduced risk
- Incremental delivery of new features
- Solving technical debt
- Define the domain area
- Techniques
- Domain-Driven Design
- Areas: domain, subdomain, context
- Design
- Strategic — Bounded Context, Context Map
- Tactical — Entities, Value Objects, Aggregates, Repositories, Services
- Domain identification
- Analyze the business and requirements
- Identify domains
- Split into subdomains
- Define contexts
- Create context maps
- Iterative refinement
- Technical analysis
- Performance analysis
- Domain-Driven Design
- Prioritization
- Criteria: business impact, technical impact, customer impact
- Methods
- MoSCoW — Must have, Should have, Could have, Won’t have
- Eisenhower matrix — urgent & important, not urgent but important, urgent but not important, not urgent & not important
- Kano model — Must-have, Performance, Excitement, Indifferent, Reverse
- Weighted scoring model
- ICE scoring — Impact, Confidence, Effort
- Value vs. Effort
- Cost of Delay
- RICE — Reach, Impact, Confidence, Effort
- WSJF — Weighted Shortest Job First
- Techniques
- Design and develop the new services
- Monitor and verify
- Application performance monitoring
- Centralized logging
- Infrastructure monitoring
- A/B testing
- Automated testing
- Switch traffic
- Routing
- Reverse proxy
- API Gateway
- Service Mesh
- Anti-corruption layer (ACL)
- Patterns: Facade, Adapter
- Advantages
- Isolation from the complexity and problems of the legacy system
- Control over request routing
- Adaptation of data to various requirements
- Encapsulation of the monolithic system
- Routing
- Parallel Run
- High-Level Design
- Branch by Abstraction
- Volume
- Velocity
- Variety
- Veracity
- Value
- Data processing
- Data lineage
- Data catalog
- Principles
- Domain-driven data contexts
- Unified infrastructure platform
- Data as a product mindset
- Self-serve infrastructure
- Federated governance