作者：来自 Elastic Gustavo Llermaly
将我们的 Jira 内容索引到 Elaasticsearch 中以创建统一的数据源并使用文档级别安全性进行搜索。
在本文中，我们将回顾 Elastic Jira 原生连接器的一个用例。我们将使用一个模拟项目，其中一家银行正在开发一款汇款应用，需要将 Jira 中的信息集成到 Elastic 中。
原生连接器允许我们从票据、任务和其他文档中获取 Elastic 集群信息，集中数据并启用高级搜索功能。
使用此连接器的主要好处是：
- Jira 中的数据与 Elasticsearch 同步。
- 访问高级搜索功能。
- 文档级安全性 (DLS) 匹配源安全性。你只能搜索 Jira 中允许你查看的内容。
步骤
- 配置 Jira 连接器
- 将文档索引到 Elasticsearch
- 查询数据
- 文档级安全性 (document level security - DLS)
配置 Jira 连接器
首先，你需要从 Jira 获取 API token 以连接到 Elasticsearch。转到此链接了解如何创建它。
将其命名为 “elastic-connector”。它应该如下所示：
获取 token 并进入 Kibana 仪表板。然后，转到原生连接器并选择 New Jira Cloud connector。
https://<YOUR_KIBANA_URL>/app/enterprise_search/content/connectors/new_connector?service_type=jira&connector_type=native用 Kibana 端点替换 YOUR_KIBANA_URL。
将连接器命名为 “bank”，然后点击 “Create and attach an index named bank”，即可创建一个同名的新索引。
完成了！现在我们需要配置我们的 Jira 数据。
由于我们不会使用自己的 SSL 证书，因此我们将保持 “Enable SSL” 关闭。
你可以在官方文档中查看每个字段的详细信息。
激活文档级安全性 (DLS)，以便你获得有权查看文档的用户和组。
正确配置连接器后，你可以继续同步数据，如下所示。从 Jira 获取数据可能需要几分钟。
- 完整内容（Full Content）：索引所有 Jira 文档。
- 增量内容（Incremental Content）：仅索引自上次完整内容同步以来的更改。
- 访问控制（Access Control）：在安全索引中索引 Jira 用户以激活 DLS。
我们可以检查连接器的概述来查看同步是否成功。
在 “Documents” 选项卡中，我们可以准确地看到使用连接器获取的数据。第一次同步的对象是：
- Projects
- Issues
- Attachments
将文档索引到 Elasticsearch
我们不仅限于跨连接器文档进行搜索。Elasticsearch 允许你使用单个查询搜索多个索引。
在我们的示例中，我们将其他文档索引到 galactic_documents 索引中，以了解搜索如何与多个数据源配合使用：
- GBFF 合规手册
- Galactic Banking App 用户指南
- 技术规格报告
但在索引之前，我们将为每个字段创建优化映射：
PUT /galactic_documents
{
"mappings": {
"properties": {
"document_id": {
"type": "keyword"
},
"title": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"content": {
"type": "text"
},
"release_date": {
"type": "date",
"format": "yyyy-MM-dd"
},
"page_count": {
"type": "integer"
},
"tags": {
"type": "keyword"
}
}
}
}配置映射后，我们现在可以索引：
POST galactic_documents/_bulk
{ "index": { "_index": "galactic_documents", "_id": "1" } }
{ "document_id": "GBFF-001", "title": "Compliance Manual of the GBFF", "content": "This document sets forth the compliance standards for intergalactic financial entities: Quantum-level data encryption to guarantee security in all transactions. Mandatory multi-factor authentication for all users and administrators. Quarterly reviews of security policies and access audit logs.", "release_date": "2024-01-01", "page_count": 5, "tags": ["compliance", "security"] }
{ "index": { "_index": "galactic_documents", "_id": "2" } }
{ "document_id": "GBFF-002", "title": "User Guide for the Galactic Banking App", "content": "Welcome to the Galactic Banking application by Interstellar Finance Corp. Here you can: Transfer galactic credits to any registered account across the Milky Way. Check your balance and manage your investments in real-time. Access interplanetary loans with ease. For your security, use multi-factor authentication each time you log in.", "release_date": "2024-01-01", "page_count": 3, "tags": ["user guide", "application"] }
{ "index": { "_index": "galactic_documents", "_id": "3" } }
{ "document_id": "GBFF-003", "title": "Technical Specifications Report - Galactic Banking Project", "content": "This report details the technical architecture of the Galactic Banking application: Microservices-based backend for scalability and performance. Secure communication protocols utilizing quantum encryption. Transaction management adapted to environments with gravity variations and time dilation.", "release_date": "2024-01-01", "page_count": 7, "tags": ["technical", "specifications", "architecture"] }查询数据
现在我们有了 Jira 对象和文档，我们可以一起搜索它们。
GET bank,galactic_documents/_search
{
"query": {
"multi_match": {
"query": "galactic moon",
"fields": [
"content",
"title",
"*description",
"*summary"
]
}
}
}查询 “galactic moon” 将为我们获取 Jira 对象和我们索引的文档：
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 1.2613049,
"hits": [
{
"_index": "bank",
"_id": "Marketing Mars-MM-2",
"_score": 1.2613049,
"_source": {
"Type": "Task",
"Custom_Fields": {
"Satisfaction": null,
"Approvals": null,
"Change reason": null,
"Epic Link": null,
"Actual end": null,
"Design": null,
"Campaign assets": null,
"Department": null,
"Story point estimate": null,
"Approver groups": null,
"[CHART] Date of First Response": null,
"Request Type": null,
"Campaign goals": null,
"Project overview key": null,
"Related projects": null,
"Campaign type": null,
"Impact": null,
"Request participants": [],
"Locked forms": null,
"Time to first response": null,
"Work category": null,
"Audience": null,
"Open forms": null,
"Details": null,
"Sprint": null,
"Stakeholders": null,
"Marketing asset type": null,
"Submitted forms": null,
"Start date": null,
"Actual start": null,
"Category": null,
"Change risk": null,
"Target start": null,
"Issue color": null,
"Parent Link": {
"hasEpicLinkFieldDependency": false,
"showField": false,
"nonEditableReason": {
"reason": "EPIC_LINK_SHOULD_BE_USED",
"message": "To set an epic as the parent, use the epic link instead"
}
},
"Format": null,
"Target end": null,
"Approvers": null,
"Team": null,
"Change type": null,
"Satisfaction date": null,
"Request language": null,
"Amount": null,
"Rank": "0|i0003j:",
"Affected services": null,
"Type": null,
"Time to resolution": null,
"Total forms": null,
"[CHART] Time in Status": null,
"Organizations": [],
"Flagged": null,
"Project overview status": null
},
"Issue": {
"statuscategorychangedate": "2024-11-01T17:52:30.550-0300",
"issuetype": {
"avatarId": 10318,
"hierarchyLevel": 0,
"name": "Task",
"self": "https://xxxx.atlassian.net/rest/api/2/issuetype/10017",
"description": "A small, distinct piece of work.",
"entityId": "f30ea676-7b3d-44ad-9858-558081742a2e",
"id": "10017",
"iconUrl": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10318?size=medium",
"subtask": false
},
"components": [],
"timespent": null,
"timeoriginalestimate": null,
"project": {
"simplified": true,
"avatarUrls": {
"48x48": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418",
"24x24": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=small",
"16x16": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=xsmall",
"32x32": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=medium"
},
"name": "Marketing Mars",
"self": "https://xxxx.atlassian.net/rest/api/2/project/10003",
"id": "10003",
"projectTypeKey": "business",
"key": "MM"
},
"description": null,
"fixVersions": [],
"aggregatetimespent": null,
"resolution": null,
"timetracking": {},
"security": null,
"aggregatetimeestimate": null,
"attachment": [],
"resolutiondate": null,
"workratio": -1,
"summary": "Conquer the moon",
"issuerestriction": {
"issuerestrictions": {},
"shouldDisplay": true
},
"watches": {
"self": "https://xxxx.atlassian.net/rest/api/2/issue/MM-2/watchers",
"isWatching": true,
"watchCount": 1
},
"lastViewed": "2024-11-01T17:52:34.925-0300",
"creator": {
"accountId": "712020:88983800-6c97-469a-9451-79c2dd3732b5",
"emailAddress": "[email protected]",
"avatarUrls": {
"48x48": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png",
"24x24": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png",
"16x16": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png",
"32x32": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png"
},
"displayName": "Tomas Murua",
"accountType": "atlassian",
"self": "https://xxxx.atlassian.net/rest/api/2/user?accountId=712020%3A88983800-6c97-469a-9451-79c2dd3732b5",
"active": true,
"timeZone": "Chile/Continental"
},
"subtasks": [],
"created": "2024-11-01T17:52:30.289-0300",
"reporter": {
"accountId": "712020:88983800-6c97-469a-9451-79c2dd3732b5",
"emailAddress": "[email protected]",
"avatarUrls": {
"48x48": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png",
"24x24": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png",
"16x16": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png",
"32x32": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png"
},
"displayName": "Tomas Murua",
"accountType": "atlassian",
"self": "https://xxxx.atlassian.net/rest/api/2/user?accountId=712020%3A88983800-6c97-469a-9451-79c2dd3732b5",
"active": true,
"timeZone": "Chile/Continental"
},
"aggregateprogress": {
"total": 0,
"progress": 0
},
"priority": {
"name": "Medium",
"self": "https://xxxx.atlassian.net/rest/api/2/priority/3",
"iconUrl": "https://xxxx.atlassian.net/images/icons/priorities/medium.svg",
"id": "3"
},
"labels": [],
"environment": null,
"timeestimate": null,
"aggregatetimeoriginalestimate": null,
"versions": [],
"duedate": null,
"progress": {
"total": 0,
"progress": 0
},
"issuelinks": [],
"votes": {
"hasVoted": false,
"self": "https://xxxx.atlassian.net/rest/api/2/issue/MM-2/votes",
"votes": 0
},
"comment": {
"total": 0,
"comments": [],
"maxResults": 0,
"self": "https://xxxx.atlassian.net/rest/api/2/issue/10018/comment",
"startAt": 0
},
"assignee": null,
"worklog": {
"total": 0,
"maxResults": 20,
"startAt": 0,
"worklogs": []
},
"updated": "2024-11-01T17:52:42.711-0300",
"status": {
"name": "To Do",
"self": "https://xxxx.atlassian.net/rest/api/2/status/10014",
"description": "",
"iconUrl": "https://xxxx.atlassian.net/",
"id": "10014",
"statusCategory": {
"colorName": "blue-gray",
"name": "To Do",
"self": "https://xxxx.atlassian.net/rest/api/2/statuscategory/2",
"id": 2,
"key": "new"
}
}
},
"id": "Marketing Mars-MM-2",
"_timestamp": "2024-11-01T17:52:42.711-0300",
"Key": "MM-2",
"_allow_access_control": [
"account_id:712020:88983800-6c97-469a-9451-79c2dd3732b5",
"name:Tomas-Murua"
]
}
},
{
"_index": "galactic_documents",
"_id": "2",
"_score": 0.61183906,
"_source": {
"document_id": "GBFF-002",
"title": "User Guide for the Galactic Banking App",
"content": "Welcome to the Galactic Banking application by Interstellar Finance Corp. Here you can: Transfer galactic credits to any registered account across the Milky Way. Check your balance and manage your investments in real-time. Access interplanetary loans with ease. For your security, use multi-factor authentication each time you log in.",
"release_date": "2024-01-01",
"page_count": 3,
"tags": [
"user guide",
"application"
]
}
},
{
"_index": "galactic_documents",
"_id": "3",
"_score": 0.5029222,
"_source": {
"document_id": "GBFF-003",
"title": "Technical Specifications Report - Galactic Banking Project",
"content": "This report details the technical architecture of the Galactic Banking application: Microservices-based backend for scalability and performance. Secure communication protocols utilizing quantum encryption. Transaction management adapted to environments with gravity variations and time dilation.",
"release_date": "2024-01-01",
"page_count": 7,
"tags": [
"technical",
"specifications",
"architecture"
]
}
}
]
}
}如果文档太长，你可以将选项 _source 添加到查询中，以仅包含你需要的字段。如果你只想删除一些字段，我们将在本系列的第二部分介绍该选项。
文档级安全性 (DLS)
我们现在将配置文档级安全性 (DLS)，以将 Jira 权限与 Elasticsearch 中的权限相匹配，以便用户搜索时只能看到他们在 Jira 中被允许看到的内容。
首先，我们将转到 Elastic Cloud 中连接器的控制面板，然后单击 Access Control Sync。
此同步将带来 Jira 用户的访问和权限信息。为了测试这一点，我创建了另一个 Jira 板（board），但用户 “Gustavo” 无权访问该板。
让我们开始检查新板中的文档是否存在：
GET bank/_search
{
"_source": ["Issue.summary"],
"query": {
"match": {
"Issue.project.name": "Marketing Mars"
}
}
}我们可以有效地看到以下问题：
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.7473189,
"hits": [
{
"_index": "bank",
"_id": "Marketing Mars-MM-1",
"_score": 0.7473189,
"_source": {
"Issue": {
"summary": "Conquer Mars"
}
}
},
{
"_index": "bank",
"_id": "Marketing Mars-MM-3",
"_score": 0.7473189,
"_source": {
"Issue": {
"summary": "Conquering Earth"
}
}
},
{
"_index": "bank",
"_id": "Marketing Mars-MM-2",
"_score": 0.7473189,
"_source": {
"Issue": {
"summary": "Conquer the moon"
}
}
}
]
}
}但是，由于用户 “Gustavo” 没有访问权限，因此他应该无法看到它们。
让我们在 ACL 过滤器索引中查找用户的文档以查看他们的权限。
GET .search-acl-filter-bank/_search响应：
{
"_index": ".search-acl-filter-bank",
"_id": "63c04b092341bff4fff6e0cb",
"_score": 1,
"_source": {
"created_at": "2024-11-01T23:19:35.784996+00:00",
"id": "63c04b092341bff4fff6e0cb",
"_timestamp": "2024-11-01T05:42:04.410478+00:00",
"identity": {
"account_id": "account_id:63c04b092341bff4fff6e0cb",
"email_address": null,
"display_name": "name:Gustavo",
"locale": "locale:en_US"
},
"query": {
"template": {
"source": """{
"bool": {
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "_allow_access_control"
}
}
}
},
{
"terms": {
"_allow_access_control.enum": {{#toJson}}access_control{{/toJson}}
}
}
]
}
}""",
"params": {
"access_control": [
"account_id:63c04b092341bff4fff6e0cb",
"group_id:d3f28403-7e99-4262-8f11-77a75bcd33d8",
"role_key:jira-software"
]
}
}
}
}
}该索引包括用户 ID 及其所有 Jira 组。我们需要将用户访问控制中的内容与每个文档中的字段 _allowed_access_control 进行匹配。
我们将使用以下命令为 Gustavo 创建 API 密钥。你必须从上一步复制 query.template 值：
POST /_security/api_key
{
"name": "gustavo",
"expiration": "30d",
"role_descriptors": {
"jira-role": {
"index": [
{
"names": [
"bank",
"galactic_documents"
],
"privileges": [
"read",
"view_index_metadata"
],
"query": {
"template": {
"params": {
"access_control": [
"account_id:63c04b092341bff4fff6e0cb",
"group_id:d3f28403-7e99-4262-8f11-77a75bcd33d8",
"role_key:jira-software"
]
},
"source": """{
"bool": {
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "_allow_access_control"
}
}
}
},
{
"terms": {
"_allow_access_control.enum": {{#toJson}}access_control{{/toJson}}
}
}
]
}
}"""
}
}
}
]
}
}
}请注意，我们仅通过此选项授予对本文中索引的访问权限。
为 Gustavo 创建 API 密钥的响应如下：
{
"id": "yLa1FJMBU4bZPaw5Stnl",
"name": "gustavo",
"expiration": 1733811245816,
"api_key": "UrGdsnDFSyGxjQvLayw5jQ",
"encoded": "eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ=="
}你可以使用 curl 来测试我们是否可以使用 API KEY 运行搜索，并且它不会从 Marketing board 获取信息，因为 Gustavo 无权访问它。
curl --location --request GET 'https://interstellar-finance-corp.es.us-central1.gcp.cloud.es.io/bank/_search' \
--header 'Authorization: ApiKey eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ==' \
--header 'Content-Type: application/json' \
--data '{
"_source": ["Issue.summary"],
"query": {
"match": {
"Issue.project.name": "Marketing Mars"
}
}
}'响应：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}我们可以看到，Gustavo 没有获得任何信息，因为他没有访问权限。现在，让我们用他被允许查看的 board 文件进行测试：
curl --location --request GET 'https://interstellar-finance-corp.es.us-central1.gcp.cloud.es.io/bank/_search?pretty=true' \
--header 'Authorization: ApiKey eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ==' \
--header 'Content-Type: application/json' \
--data '{
"_source": ["Issue.summary"],
"query": {
"match": {
"Issue.project.name": "Galactic Banking Project"
}
}
}'响应：
{
"took" : 7,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 3.1784885,
"hits" : [
{
"_index" : "bank",
"_id" : "Galactic Banking Project-GBP-3",
"_score" : 3.1784885,
"_source" : {
"Issue" : {
"summary" : "Intergalactic Security and Compliance"
}
}
},
{
"_index" : "bank",
"_id" : "Galactic Banking Project-GBP-2",
"_score" : 0.5469647,
"_source" : {
"Issue" : {
"summary" : "Bank Application Frontend"
}
}
},
{
"_index" : "bank",
"_id" : "Galactic Banking Project-GBP-1",
"_score" : 0.5469647,
"_source" : {
"Issue" : {
"summary" : "Development of API for International Transfers"
}
}
}
]
}
}结论
如你所见，将 Elasticsearch 与 Jira 集成有许多好处，例如能够对你正在处理的所有项目进行统一搜索，以及能够在多个数据源中运行更高级的搜索。添加的 DLS 是一种快速简便的方法，可确保用户保持他们在原始源中已有的访问权限。
想要获得 Elastic 认证？了解下一次 Elasticsearch 工程师培训何时开始！
Elasticsearch 包含新功能，可帮助你为你的用例构建最佳搜索解决方案。深入了解我们的示例笔记本以了解更多信息，开始免费云试用，或立即在你的本地机器上试用 Elastic。