Introduction xxiii
Chapter 1 Overview of Generative Artificial Intelligence Security 1
Common Use Cases for Generative AI in the Enterprise 1
Generative Artificial Intelligence 1
Generative AI Use Cases 2
LLM Terminology 3
Sample Three-Tier Application 4
Presentation Tier 5
Application Tier 5
Data Tier 5
Generative AI Application Risks 5
Hallucinations 6
Malicious Usage 6
Shadow AI 7
Unfavorable Business Decisions 8
Established Risks 8
Shared AI Responsibility Model 8
Shared Responsibility Model for the Cloud 9
Shared Responsibility Model for AI 10
AI Usage 10
AI Application 10
AI Platform 11
Applying the Shared Responsibility Model 11
Regulation and Control Frameworks 12
Regulation in the United States 12
Regulation in the European Union 12
NIST AI Risk Management Framework 14
Govern 15
Map 15
Measure 16
Manage 16
Key Takeaways 16
References 17
Chapter 2 Security Controls for Azure OpenAI Service 19
On the Importance of Selecting Appropriate Security Controls 19
Risk Appetite 20
Comparing OpenAI Hosting Models 21
OpenAI ChatGPT 21
Privacy and Compliance 21
Identity and Access Management 21
Data Protection and Encryption 22
Audit Logging 22
Network Isolation 22
Data Residency 22
Azure OpenAI 22
Privacy and Compliance 23
Identity and Access Management 23
Data Protection and Encryption 23
Audit Logging 23
Network Isolation 23
Data Residency 23
Recommendation for Enterprise Usage 24
Evaluating Security Controls with MCSB 24
Control Domains 26
Network Security 27
Identity Management 28
Privileged Access 28
Data Protection 29
Asset Management 29
Logging and Threat Detection 29
Incident Response 30
Posture and Vulnerability Management 30
Endpoint Security 31
Backup and Recovery 31
DevOps Security 32
Governance and Strategy 32
Security Baselines 33
Applying Microsoft Cloud Security Baseline to Azure OpenAI 33
Security Profile 34
How to Approach the Security Baseline 34
Data Protection 35
Identity Management 36
Logging and Threat Detection 37
Network Security 38
Asset Management 38
Backup and Recovery 39
Endpoint Security 40
Posture and Vulnerability Management 40
Privileged Access 41
Selected Controls 42
Mapping the Selected Controls to CIS and NIST 44
Using Azure Policy to Secure Azure OpenAI at Scale 46
Azure Policy 46
Continuous Compliance Monitoring 47
Azure Policies for Azure OpenAI 48
Key Takeaways 49
References 49
Chapter 3 Implementing Azure OpenAI Security Controls 51
OWASP Top 10 for LLM Applications 51
Prompt Injection 52
Insecure Output Handling 52
Training Data Poisoning 53
Model Denial of Service 53
Supply Chain Vulnerabilities 53
Sensitive Information Disclosure 54
Insecure Plugin Design 54
Excessive Agency 54
Overreliance 55
Model Theft 55
Access Control 56
Implementing Access Control for Azure OpenAI 56
Cognitive Services OpenAI User 57
Cognitive Services OpenAI Contributor 58
Azure AI Administrator 59
Azure AI Developer 61
Azure AI Enterprise Network Connection Approver 62
Azure AI Inference Deployment Operator 64
Preventing Local Authentication 65
Disable Local Authentication Using Bicep 66
Disable Local Authentication Using Terraform 66
Disable Local Authentication Using ARM Templates 67
Prevent Local Authentication Using PowerShell 67
Enforcing with Azure Policy 67
Audit Logging 68
Control Plane Audit Logging 68
Data Plane Audit Logging 71
Enable Data Plane Audit Logging Using Azure Portal 72
Enable Data Plane Audit Logging Using Bicep 73
Enable Data Plane Audit Logging Using Terraform 73
Enable Data Plane Audit Logging Using ARM Templates 74
Enable Data Plane Audit Logging Using PowerShell 76
Enable Data Plane Audit Logging Using Azure cli 76
Enforcing with Azure Policy 77
Enable Logging by Category Group for Cognitive Services 77
Network Isolation 82
Default Network Controls 83
Control Inbound Network Traffic 83
Control Inbound Network Traffic Using the Azure Portal 84
Control Inbound Network Traffic Using Bicep 84
Control Inbound Network Traffic with Private Endpoints Using Infrastructure as Code 85
Control Inbound Network Traffic Using Terraform 87
Control Inbound Network Traffic with Private Endpoints Using Terraform 87
Control Inbound Network Traffic Using ARM Templates 89
Control Inbound Network Traffic with Private Endpoints Using ARM Templates 90
Control Inbound Network Traffic Using PowerShell 93
Control Inbound Network Traffic with Private Endpoints Using PowerShell 94
Control Inbound Network Traffic Using Azure cli 95
Control Inbound Network Traffic with Private Endpoints Using Azure cli 95
Control Outbound Network Traffic 97
Enable Data Loss Prevention Using REST 97
Enable Data Loss Prevention Using Bicep 98
Enable Data Loss Prevention Using Terraform 98
Enable Data Loss Prevention Using ARM Templates 99
Enforcing with Azure Policy 101
Azure AI Services Resources Should Restrict Network Access 101
Azure AI Services Resources Should Use Azure Private Link 103
Encryption at Rest 105
Implementing Azure OpenAI with CMK 106
Implement CMK Using Azure Portal 106
Implement CMK Using Bicep 107
Implement CMK Using Terraform 109
Implement CMK Using ARM Templates 111
Implement CMK Using PowerShell 114
Implement CMK Using the Azure cli 115
Enforcing with Azure Policy 116
Azure AI Services Resources Should Encrypt Data at Rest with a CMK 117
Content Filtering Controls 119
System Safety Prompts 119
Azure AI Content Safety 120
Content Filtering 120
Prompt Shields 121
Protected Material Detection 121
Groundedness Detection 121
Creating a Content Filter 121
Implementing Content Filtering Programmatically 122
Content Safety Input Restrictions 123
Key Takeaways 123
References 124
Chapter 4 Securing the Entire Application 125
The Three-Tier LLM Application in Azure 125
Presentation Tier 126
Application Tier 126
Data Tier 126
On Threat Modeling 126
Threat Model of the Three-Tier Application 127
Revised Application Architecture 129
Retrieval-Augmented Generation 129
RAG in Azure 130
Azure AI Search 130
Azure Cosmos DB 131
Application Architecture with RAG 131
Azure Front Door 132
Security Profile 132
Security Baseline 132
Implementing Security Controls 133
Access Control 133
Audit Logging 133
Network Isolation 141
Encryption at Rest 152
Enforcing Controls with Policies 152
Azure App Service 153
Security Profile 153
Security Baseline 153
Implementing Security Controls 155
Access Control 156
Audit Logging 163
Network Isolation 169
Encryption at Rest 176
Enforcing Controls with Policies 176
API Management 177
Security Profile 177
Security Baseline 178
Implementing Security Controls 178
Access Control 179
Audit Logging 180
Network Isolation 186
Encryption at Rest 201
Enforcing Controls with Policies 202
Storage Account 202
Security Profile 202
Security Baseline 203
Implementing Security Controls 204
Access Control 204
Audit Logging 209
Network Isolation 216
Encryption at Rest 225
Backup and Recovery 232
Discover, Classify, and Protect Sensitive Data 238
Enforcing Controls with Policies 238
Cosmos DB 238
Security Profile 239
Security Baseline 239
Implementing Security Controls 241
Access Control 241
Audit Logging 244
Network Isolation 249
Encryption at Rest 256
Backup and Recovery 262
Enforcing Controls with Policies 266
Azure AI Search 266
Security Profile 266
Security Baseline 267
Implementing Security Controls 268
Access Control 268
Audit Logging 272
Network Isolation 278
Encryption at Rest 287
Enforcing Controls with Policies 294
Key Takeaways 294
References 294
Chapter 5 Moving to Production 297
LLM Application Security Lifecycle 297
Model Supply Chain 298
Security Testing 299
Model Safety Evaluation 299
How to Use Model Safety Evaluation 300
Adversarial Testing 300
How to Use the Adversarial Simulator Service 301
Red Teaming 304
Crescendo Multiturn Attack 304
Red Teaming with PyRIT 304
Content Credentials 305
AI Security Posture Management 307
Discover and Manage Shadow AI 307
Discover SaaS Applications 307
Discover Generative AI Applications 309
Manage Generative AI Applications 312
Alert on Anomalous Activity and Applications 313
Defender for Cloud AI Workloads 314
Discovery 314
Posture Management 314
Security Alerting 314
Security Posture Management 315
Investigating Security Alerts 316
Alert Details 317
Supporting Evidence 318
Take Action 319
Managing Incidents 323
Instrumenting Security Alert Ingestion 324
Azure OpenAI Alerts 326
Detected Credential Theft Attempts on an Azure OpenAI Model Deployment 327
A Jailbreak Attempt on an Azure OpenAI Model Deployment Was Blocked by Azure AI Content Safety Prompt Shields 327
A Jailbreak Attempt on an Azure OpenAI Model Deployment Was Detected by Azure AI Content Safety Prompt Shields 327
Sensitive Data Exposure Detected in Azure OpenAI Model Deployment 327
Corrupted AI Application, Model, or Data Directed a Phishing Attempt at a User 328
Phishing URL Shared in an AI Application 328
Phishing Attempt Detected in an AI Application 328
Defender for Cloud Alerts for Other Services 328
App Service Alerts 329
API Management Alerts 330
Storage Account Alerts 331
Cosmos DB Alerts 332
LLM Application in Your Cloud Security Architecture 332
Cloud Security Control Domains 333
Asset Management 333
Incident Response 334
Privileged Access 336
Posture and Vulnerability Management 337
Landing Zones 339
About Landing Zones 339
Microsoft Enterprise-Scale Landing Zones 339
Microsoft Landing Zone Accelerator for OpenAI 342
LLM Application in the Landing Zone 342
The Sample Application in the Landing Zone 342
Access Control 343
Security Monitoring 343
Incident Response 344
Network 344
Key Takeaways 345
References 345
Index 347
