pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils

Module Contents

Classes

CodeLineIterator

Built-in mutable sequence.

Functions

random_indices(source, percentage)
_switch_token(tokens, ids)
_replace_token(tokens, ids)
_delete_token(tokens, ids)
_add_token(tokens, ids)
_prepare_corrupt_code(code_src)
remove_comment(code_str[, tokenizer])	Remove comments from code string,
read_defect_examples(lines, data_num[, ...])	Read examples from filename.
calc_stats(examples[, tokenizer, is_tokenize])

class pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils.CodeLineIterator(code, strip=True)

Bases: list

Built-in mutable sequence.

If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.

__getitem__(item)

x.__getitem__(y) <==> x[y]

__setitem__(key, value)

Set self[key] to value.

__iter__()

Implement iter(self).

__len__()

Return len(self).

__str__()

Return str(self).

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils.random_indices(source, percentage)

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils._switch_token(tokens: list, ids: list)

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils._replace_token(tokens: list, ids: list)

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils._delete_token(tokens: list, ids: list)

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils._add_token(tokens: list, ids: list)

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils._prepare_corrupt_code(code_src)

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils.remove_comment(code_str, tokenizer=None)

Remove comments from code string, :param code_str: code string :param tokenizer: tokenizer if passed, will add <mask> token to the code

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils.read_defect_examples(lines, data_num, remove_comments=True, tokenizer=None)

Read examples from filename.

pyabsa.tasks.CodeDefectDetection.dataset_utils.cdd_utils.calc_stats(examples, tokenizer=None, is_tokenize=False)