COLLECTED BY

Organization: Archive Team

Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.

History is littered with hundreds of conflicts over the future of a community, group, location or business that were "resolved" when one of the parties stepped ahead and destroyed what was there. With the original point of contention destroyed, the debates would fall to the wayside. Archive Team believes that by duplicated condemned data, the conversation and debate can continue, as well as the richness and insight gained by keeping the materials. Our projects have ranged in size from a single volunteer downloading the data to a small-but-critical site, to over 100 volunteers stepping forward to acquire terabytes of user-created data to save for future generations.

The main site for Archive Team is at archiveteam.org and contains up to the date information on various projects, manifestos, plans and walkthroughs.

This collection contains the output of many Archive Team projects, both ongoing and completed. Thanks to the generous providing of disk space by the Internet Archive, multi-terabyte datasets can be made available, as well as in use by the Wayback Machine, providing a path back to lost websites and work.

Our collection has grown to the point of having sub-collections for the type of data we acquire. If you are seeking to browse the contents of these collections, the Wayback Machine is the best first stop. Otherwise, you are free to dig into the stacks to see what you may find.

The Archive Team Panic Downloads are full pulldowns of currently extant websites, meant to serve as emergency backups for needed sites that are in danger of closing, or which will be missed dearly if suddenly lost due to hard drive crashes or server failures.

Collection: Archive Team: URLs

TIMESTAMPS

The Wayback Machine - https://web.archive.org/web/20210414125600/https://github.com/topics/text-processing

#

text-processing

Here are 905 public repositories matching this topic...

learnbyexample / Command-line-text-processing

Star

⚡ From finding text to search and replace, from sorting to beautifying text and more 🎨

ruby linux command-line regex perl ebook awk sed text-processing grep

Updated Dec 13, 2020
Shell

google / diff-match-patch

Star

Diff Match Patch is a high-performance library in multiple languages that manipulates plain text.

diff match patch text-processing difference

Updated Mar 4, 2021
Python

chmln / sd

Star

Intuitive find & replace CLI (sed alternative)

rust terminal command-line regex text-processing

Updated Mar 28, 2021
Rust

fastnlp / fastNLP

Star

fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.

natural-language-processing deep-learning text-classification chinese-nlp text-processing nlp-parsing nlp-library

Updated Apr 12, 2021
Python

kk7nc / Text_Classification

Star

Text Classification Algorithms: A Survey

deep-learning random-forest text-classification recurrent-neural-networks naive-bayes-classifier dimensionality-reduction logistic-regression document-classification convolutional-neural-networks text-processing decision-trees boosting-algorithms support-vector-machines hierarchical-attention-networks nlp-machine-learning conditional-random-fields k-nearest-neighbours deep-belief-network rocchio-algorithm deep-neural-network

Updated Apr 9, 2021
Python

pyparsing / pyparsing

Star

Python library for creating PEG parsers

python parsing parser-combinators python3 parsing-expression-grammar python-3 text-processing python-2 python2 parsing-library peg-parsers

Updated Apr 13, 2021
Python

abadojack / whatlanggo

Star

Natural language detection library for Go

nlp go language text-processing

Updated Jan 15, 2021
Go

derek73 / python-nameparser

Star

A simple Python module for parsing human names into their individual components

python text-processing text-parser python-module

Updated Mar 25, 2021
Python

cbaziotis / ekphrasis

Star

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

nlp tokenizer text-processing semeval nlp-library word-segmentation spelling-correction tokenization text-segmentation spell-corrector word-normalization

Updated Feb 8, 2021
Python

open-korean-text / open-korean-text

Star

Open Korean Text Processor - An Open-source Korean Text Processor

natural-language-processing tokenizer korean text-processing korean-text-processing korean-tokenizer

Updated Mar 1, 2021
Scala

BurntSushi / aho-corasick

Star

A fast implementation of Aho-Corasick in Rust.

search finite-state-machine text-processing aho-corasick substring-matching

Updated Nov 3, 2020
Rust

proycon / pynlpl

Star

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

python nlp machine-learning natural-language-processing library linguistics computational-linguistics text-processing nlp-library search-algorithms evaluation-metrics folia language-modelling

Updated Mar 13, 2019
Python

andrewbihl / bsed

Star

Open

Allow catch-all line commands

andrewbihl commented Jan 27, 2020

I'd like to be able to run commands on all lines of a file. For example, bsed wrap lines with " should execute on all lines of the file. Current workaround is to include some trivial filter like wrap lines containing '.' with "

Read more

Easy good first issue

airbnb / artificial-adversary

Star

🗣️ Tool to generate adversarial text examples and test machine learning models against them

python spam data-science machine-learning text-mining data-mining text-classification metrics text text-analysis python3 classification text-processing python2 spam-filtering spam-detection spam-classification adversarial-examples black-box-attacks black-box-benchmarking

Updated Oct 14, 2018
Python

textpipe / textpipe

Star

Open

Add installation instructions in README file

2

dungchu commented Jul 4, 2018

pip install -r requirements.txt
python -m spacy download en_core_web_sm # depend on which language you want to you: https://spacy.io/usage/models

Read more

enhancement good first issue

gagolews / stringi

Star

THE String Processing Package for R (with ICU)

nlp unicode natural-language-processing r text icu regex regexp tidy-data string-manipulation text-processing stringr stringi icu4c

Updated Apr 7, 2021
C++

BurntSushi / regex-automata

Star

A low level regular expression library that uses deterministic finite automata.

rust automata regex regexp text-processing nfa automaton dfa regex-engine

Updated Jan 20, 2021
Rust

rust-unic

open-i18n / rust-unic

Star

Open

Support for stdin

2

ad-si commented Jan 8, 2019

This should work:

cat test.txt | unic-inspector

Read more

A: apps enhancement good first issue help wanted

pyarabic

linuxscout / pyarabic

Star

pyarabic

text-processing nlp-library arabic-language

Updated Sep 2, 2020
Python

s3nh / text-detector

Sponsor Star

Tool which allow you to detect and translate text.

nlp recognition deep-learning text craft pytorch text-recognition text-processing ocr-recognition crnn scene-text-detection scene-text-detectors

Updated Sep 10, 2019
Python

textvec / textvec

Star

Text vectorization tool to outperform TFIDF for classification tasks

python nlp machine-learning natural-language-processing text-classification text-analysis tf-idf text-processing

Updated Dec 3, 2020
Python

ikegami-yukino / jaconv

Sponsor Star

Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku

transliteration japanese-language text-processing pure-python preprocessing character-converter japanese-kana julius

Updated Apr 2, 2021
Python

NIHOPA / NLPre

Star

Python library for Natural Language Preprocessing (NLPre)

python nlp natural-language-processing text-processing nlp-parsing

Updated Nov 15, 2019
Python

hakatashi / japanese.js

Star

Util collection for Japanese text processing. Hiraganize, Katakanize, and Romanize.

javascript utility katakana hiragana japanese text-processing romanize

Updated Aug 27, 2020
JavaScript

assafmo / xioc

Star

Extract indicators of compromise from text, including "escaped" ones.

ioc text-mining data-mining command-line regex regexp extract extraction command-line-tool text-processing iocs defang indicators-of-compromise escaping

Updated Apr 19, 2020
Go

microsoft / browsecloud

Star

A web app to create and browse text visualizations for automated customer listening.

visualization nlp text-classification text-processing bayesian-networks counting-grids

Updated Mar 30, 2021
TypeScript

stanfordnlp / stanza-old

Star

Stanford NLP group's shared Python tools.

python nlp natural-language-processing text-analysis text-processing

Updated Mar 14, 2018
Python

WZBSocialScienceCenter / tmtoolkit

Star

Text Mining and Topic Modeling Toolkit for Python with parallel processing power

python nlp evaluation topic-modeling text-processing parallel-processing socialscience

Updated Aug 11, 2020
Python

himkt / konoha

Sponsor Star

🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.

nlp natural-language-processing japanese text-processing mecab kytea sudachi allennlp sentencepiece janome

Updated Apr 4, 2021
Python

lyeoni / prenlp

Star

Preprocessing Library for Natural Language Processing

nlp natural-language-processing text-processing text-preprocessing preprocessing-library

Updated Feb 5, 2020
Python

Improve this page

Add a description, image, and links to the text-processing topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the text-processing topic, visit your repo's landing page and select "manage topics."