What is data integration?

Data integration is the central element in the pre-processing of data, aiming to scale up data in different units to a specific range (e.g. 0 to 1) or distribution (e.g. standard normal distribution).

In machine learning, most algorithms (e.g. SVM, KNN, Neuronet) are very sensitive to the scale of data. If the range of values for one feature is [0, 1], while the other feature is [0,10000], the model may over-regulate the larger value characteristics, leading to a deviation in the projection results. By integrating, we can balance the weight of the characteristics and accelerate the pace of the reduction of the gradient.

Common algorithms detail

Min-Max Normalization: Also known as the separation standardization. It maps the original data linearly between [0,1]. Formula is: x'=(x-min)/ (max-min). It applies to scenarios that are strictly required at borders and that have a more even distribution of data.
Z-Score Standardization: Scaled on average (mean) and standard deviation based on raw data. processed data corresponds to the distribution of the mean 0 and standard difference 1 respectively. Formula is: x'=(x-m)/ spec. The scenario that applies to data containing anomalies (Outliers) or algorithms assuming that data are distributed correctly.
Decimal Scaling: Scale by moving the decimal point. The number of digits moved depends on the number of absolute values in the data.

Why select a tool for the integration of KiloCell of data?

Our tools provide the most visible and efficient treatment experience:

Privacy and security: All calculation logic is done locally in your browser, and the data will never be uploaded to any server.
First-click statistics: Processing of the core statistical indicators of average, standard deviation, polar values, etc., automatically.
Multiformat support: Supports original data input separated by commas, spaces separated or line breaks.
ResultExport: Supports the export of the processed results to a CSV file with UTF-8 BOM, perfect compatible with Excel.