ToolsTokenize Method (String)

Splits the given text into individual atomic words, irrespective of punctuation and other marks.

Namespace: Accord.MachineLearning
Assemblies: Accord.MachineLearning (in Accord.MachineLearning.dll) Version: 3.8.0
Accord.Text (in Accord.Text.dll) Version: 3.8.0

Syntax

Copy

public static string[] Tokenize(
	this string x
)

<ExtensionAttribute>
Public Shared Function Tokenize ( 
	x As String
) As String()

Request Example View Source

Parameters

x: Type: SystemString

Return Value

Type: String

Usage Note

In Visual Basic and C#, you can call this method as an instance method on any object of type String. When you use instance method syntax to call this method, omit the first parameter. For more information, see Extension Methods (Visual Basic) or Extension Methods (C# Programming Guide).

Reference

Tools Class

Tokenize Overload

Accord.MachineLearning Namespace

ToolsAddLanguageSpecificTextSet("LST2848D4F7_0?cpp=::|nu=.");Tokenize Method (String)

Parameters

Return Value

Usage Note

Reference

ToolsTokenize Method (String)