[LakeFormation] python boto3 list_permissions 기반 모든 permissions 추출

2023년 05월 18일 17시 10분 03초에 업로드 된 글입니다.

작성자: DE 군고구마

안녕하세요.

주형권입니다. 이직을 하고 최근에 AWS 환경에서 DataLake를 구축하다보니 AWS 기반의 작업을 많이 하고 있습니다. 저 같은 경우 회사를 옮기면 우선 모니터링부터 만들고 시작하니, 초반에 모니터링에 관련 된 여러가지 글을 작성하게 되었습니다. 이 글에서 나오는 내용은 Python의 boto3의 LakeFormation의 list_permissions를 이용하였습니다.

추출 결과값

Name(Kor)	boto3 dict result Key Name	describe
유저 ID (arn)	DataLakePrincipalIdentifier (string)	An identifier for the Lake Formation principal.
권한 리스트	Permissions (list)	The permissions to be granted or revoked on the resource.
권한 리스트	PermissionsWithGrantOption (list)	Indicates whether to grant the ability to grant permissions (as a subset of permissions granted).
리소스 타입	resource type	리소스 타입 (임의 추가) DATABASE \| TABLE \| COLUMN
카탈로그 ID	CatalogId (string) –	The identifier for the Data Catalog. By default, it is the account ID of the caller.
데이터베이스명	DatabaseName (string) –	The name of the database for the table. Unique to a Data Catalog. A database is a set of associated table definitions organized into a logical group. You can Grant and Revoke database privileges to a principal.
테이블명	Name (string) –	The name of the table.
테이블 와일드 카드	TableWildcard (dict) –	A wildcard object representing every table under a database. At least one of TableResource$Name or TableResource$TableWildcard is required.
컬럼 와일드 카드	ColumnWildcard (dict) –	A wildcard specified by a ColumnWildcard object. At least one of ColumnNames or ColumnWildcard is required.

위의 항목만 따로 추출하였고, Print하여 결과를 보시면 위의 값들만 나옵니다. 필요에 따라서 원하시는 결과를 찾아서 쓰시면 좋을거 같습니다. 아래는 참고를 위해서 Response Syntax를 첨부 하였습니다.

{
    'PrincipalResourcePermissions': [
        {
            'Principal': {
                'DataLakePrincipalIdentifier': 'string'
            },
            'Resource': {
                'Catalog': {},
                'Database': {
                    'CatalogId': 'string',
                    'Name': 'string'
                },
                'Table': {
                    'CatalogId': 'string',
                    'DatabaseName': 'string',
                    'Name': 'string',
                    'TableWildcard': {}
                },
                'TableWithColumns': {
                    'CatalogId': 'string',
                    'DatabaseName': 'string',
                    'Name': 'string',
                    'ColumnNames': [
                        'string',
                    ],
                    'ColumnWildcard': {
                        'ExcludedColumnNames': [
                            'string',
                        ]
                    }
                },
                'DataLocation': {
                    'CatalogId': 'string',
                    'ResourceArn': 'string'
                },
                'DataCellsFilter': {
                    'TableCatalogId': 'string',
                    'DatabaseName': 'string',
                    'TableName': 'string',
                    'Name': 'string'
                },
                'LFTag': {
                    'CatalogId': 'string',
                    'TagKey': 'string',
                    'TagValues': [
                        'string',
                    ]
                },
                'LFTagPolicy': {
                    'CatalogId': 'string',
                    'ResourceType': 'DATABASE'|'TABLE',
                    'Expression': [
                        {
                            'TagKey': 'string',
                            'TagValues': [
                                'string',
                            ]
                        },
                    ]
                }
            },
            'Permissions': [
                'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ASSOCIATE',
            ],
            'PermissionsWithGrantOption': [
                'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'DESCRIBE'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS'|'CREATE_TAG'|'ASSOCIATE',
            ],
            'AdditionalDetails': {
                'ResourceShare': [
                    'string',
                ]
            }
        },
    ],
    'NextToken': 'string'
}

추출 Python 코드

import boto3

aws_catalogid = 'aws account id'
region_name = "aws region"
access_key_id = 'aws access key'
secret_access_key = 'aws secret key'

lakeformation_client = boto3.client('lakeformation',
            region_name=region_name,
            aws_access_key_id=access_key_id,
            aws_secret_access_key=secret_access_key)

permission_list = []

# 첫번째 내용을 넣고 Next Token을 받는다
first_result = lakeformation_client.list_permissions(CatalogId=aws_catalogid)
first_permissions = first_result.get('PrincipalResourcePermissions')
next_token = first_result.get('NextToken')   

for user in first_permissions:
    user_name = user.get('Principal').get('DataLakePrincipalIdentifier')
    permissions_list = str(user.get('Permissions'))
    permissionswithgrantoption = str(user.get('PermissionsWithGrantOption'))
    resource = user.get('Resource')
    catalogid = ''
    database_name = ''
    table_name = ''
    tablewildcard = ''
    columnwildcard = ''
    resource_type = ''

    if user.get('Resource').get('Table') != None :
        catalogid = user.get('Resource').get('Table').get('CatalogId')
        database_name = user.get('Resource').get('Table').get('DatabaseName')
        table_name = user.get('Resource').get('Table').get('Name')
        tablewildcard = str(user.get('Resource').get('Table').get('TableWildcard'))
        columnwildcard = str(user.get('Resource').get('Table').get('ColumnWildcard'))
        resource_type = 'Table'

    if user.get('Resource').get('Database') != None  :
        catalogid = user.get('Resource').get('Database').get('CatalogId')
        database_name = user.get('Resource').get('Database').get('Name')
        table_name = None
        tablewildcard = None
        columnwildcard = None
        resource_type = 'Database'

    if user.get('Resource').get('TableWithColumns') != None :
        catalogid = user.get('Resource').get('TableWithColumns').get('CatalogId')
        database_name = user.get('Resource').get('TableWithColumns').get('DatabaseName')
        table_name = user.get('Resource').get('TableWithColumns').get('Name')
        tablewildcard = str(user.get('Resource').get('TableWithColumns').get('TableWildcard'))
        columnwildcard = str(user.get('Resource').get('TableWithColumns').get('ColumnWildcard'))
        resource_type = 'Columns'

    permission_list.append((user_name,permissions_list,permissionswithgrantoption,resource_type,catalogid,database_name,table_name,tablewildcard,columnwildcard))


while next_token != None:
    result = lakeformation_client.list_permissions(CatalogId=aws_catalogid,NextToken=next_token)
    permissions = result.get('PrincipalResourcePermissions')
    next_token = result.get('NextToken')
    for user in permissions:
        user_name = user.get('Principal').get('DataLakePrincipalIdentifier')
        permissions_list = str(user.get('Permissions'))
        permissionswithgrantoption = str(user.get('PermissionsWithGrantOption'))
        catalogid = ''
        database_name = ''
        table_name = ''
        tablewildcard = ''
        columnwildcard = ''
        resource_type = ''

        if user.get('Resource').get('Table') != None :
            catalogid = user.get('Resource').get('Table').get('CatalogId')
            database_name = user.get('Resource').get('Table').get('DatabaseName')
            table_name = user.get('Resource').get('Table').get('Name')
            tablewildcard = str(user.get('Resource').get('Table').get('TableWildcard'))
            columnwildcard = str(user.get('Resource').get('Table').get('ColumnWildcard'))
            resource_type = 'Table'

        if user.get('Resource').get('Database') != None :
            catalogid = user.get('Resource').get('Database').get('CatalogId')
            database_name = user.get('Resource').get('Database').get('Name')
            table_name = None
            tablewildcard = None
            columnwildcard = None
            resource_type = 'Database'

        if user.get('Resource').get('TableWithColumns') != None :
            catalogid = user.get('Resource').get('TableWithColumns').get('CatalogId')
            database_name = user.get('Resource').get('TableWithColumns').get('DatabaseName')
            table_name = user.get('Resource').get('TableWithColumns').get('Name')
            tablewildcard = str(user.get('Resource').get('TableWithColumns').get('TableWildcard'))
            columnwildcard = str(user.get('Resource').get('TableWithColumns').get('ColumnWildcard'))
            resource_type = 'Columns'

        permission_list.append((user_name,permissions_list,permissionswithgrantoption,resource_type,catalogid,database_name,table_name,tablewildcard,columnwildcard))

print(permission_list)

감사합니다.

참고

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lakeformation/client/list_permissions.html

list_permissions - Boto3 1.26.135 documentation

Previous list_lf_tags

boto3.amazonaws.com

저작자표시 비영리 변경금지

다음글이 없습니다.

이전글이 없습니다.

추출 결과값

추출 Python 코드

참고

티스토리툴바