# 多類 Logistic 回歸分類

James McCaffrey

``````5.33  -4.89  0.15  -6.67  0.00  1.00  0.00
``````

F 功能與 c 類的多類 LR 模型會有 (f * c) 權重和偏置 c。這些都是必須確定的數位常量。對於這個演示中，有 4 * 3 = 12 權重和 3 偏置。培訓是估計值的權重和偏置的過程。培訓是一個反覆運算過程和演示將訓練反覆運算 （通常稱為毫升文學的時代） 的最大數目設置為 100。 用於訓練多類 LR 分類器的技術被稱為批次處理梯度下降。這種技術需要叫學習速率和重量衰減率兩個參數的值。這兩個值通常可以找到試驗和錯誤，並演示分配值 0.01 和 0.10，分別。

## 理解多類 Logistic 迴歸分析

``````z0 = (w00)(x0) + (w01)(x1) + (w02)(x2) + b0
y0 = 1.0 / (1.0 + e^-z0)
z1 = (w10)(x0) + (w11)(x1) + (w12)(x2) + b1
y1 = 1.0 / (1.0 + e^-z1)
z2 = (w20)(x0) + (w21)(x1) + (w22)(x2) + b2
y2 = 1.0 / (1.0 + e^-z2)
``````

## 演示的程式結構

``````using System;
{
class LogisticMultiProgram
{
static void Main(string[] args)
{
Console.WriteLine("Begin classification demo");
...
Console.WriteLine("End demo");
}
public static void ShowData(double[][] data,
int numRows, int decimals, bool indices) { . . }
public static void ShowVector(double[] vector,
int decimals, bool newLine) { . . }
static double[][] MakeDummyData(int numFeatures,
int numClasses, int numRows, int seed) { . . }
static void SplitTrainTest(double[][] allData,
double trainPct, int seed, out double[][] trainData,
out double[][] testData) { . . }
}
public class LogisticMulti
{
private int numFeatures;
private int numClasses;
private double[][] weights; // [feature][class]
private double[] biases;    // [class]
public LogisticMulti(int numFeatures,
int numClasses) { . . }
private double[][] MakeMatrix(int rows,
int cols) { . . }
public void SetWeights(double[][] wts,
double[] b) { . . }
public double[][] GetWeights() { . . }
public double[] GetBiases() { . . }
private double[] ComputeOutputs(double[] dataItem) { . . }
public void Train(double[][] trainData, int maxEpochs,
double learnRate, double decay) { . . }
public double Error(double[][] trainData) { . . }
public double Accuracy(double[][] trainData) { . . }
private static int MaxIndex(double[] vector) { . . }
private static int MaxIndex(int[] vector) { . . }
private int[] ComputeDependents(double[] dataItem) { . . }
}
}
``````

LogisticMultiProgram 類包含 MakeDummyData、 SplitTrainTest、 ShowData 和 ShowVector 的説明器方法。這些方法創建和顯示的綜合資料。所有分類邏輯都包含在一個名為 LogisticMulti 的程式定義的類。

Main 方法創建合成資料包含這些語句：

``````int numFeatures = 4;
int numClasses = 3;
int numRows = 1000;
double[][] data = MakeDummyData(numFeatures,
numClasses, numRows, 0);
``````

``````double[][] trainData;
double[][] testData;
SplitTrainTest(data, 0.80, 7, out trainData, out testData);
ShowData(trainData, 3, 2, true);
ShowData(testData, 3, 2, true);
``````

``````LogisticMulti lc = new LogisticMulti(numFeatures, numClasses);
int maxEpochs = 100;
double learnRate = 0.01;
double decay = 0.10;
lc.Train(trainData, maxEpochs, learnRate, decay);
``````

``````double[][] bestWts = lc.GetWeights();
double[] bestBiases = lc.GetBiases();
ShowData(bestWts, bestWts.Length, 3, true);
ShowVector(bestBiases, 3, true);
``````

``````double trainAcc = lc.Accuracy(trainData, weights);
Console.WriteLine(trainAcc.ToString("F4"));
double testAcc = lc.Accuracy(testData, weights);
Console.WriteLine(testAcc.ToString("F4"));
``````

## 實施多類 LR 培訓

LogisticMulti 類的建構函式定義如下：

``````public LogisticMulti(int numFeatures, int numClasses)
{
this.numFeatures = numFeatures;
this.numClasses = numClasses;
this.weights = MakeMatrix(numFeatures, numClasses);
this.biases = new double[numClasses];
}
``````

``````private double[] ComputeOutputs(double[] dataItem)
{
double[] result = new double[numClasses];
for (int j = 0; j < numClasses; ++j) // compute z
{
for (int i = 0; i < numFeatures; ++i)
result[j] += dataItem[i] * weights[i][j];
result[j] += biases[j];
}
for (int j = 0; j < numClasses; ++j) // 1 / 1 + e^-z
result[j] = 1.0 / (1.0 + Math.Exp(-result[j]));
double sum = 0.0; // softmax scaling
for (int j = 0; j < numClasses; ++j)
sum += result[j];
for (int j = 0; j < numClasses; ++j)
result[j] = result[j] / sum;
return result;
}
``````

``````private int[] ComputeDependents(double[] dataItem)
{
double[] outputs = ComputeOutputs(dataItem); // 0.0 to 1.0
int maxIndex = MaxIndex(outputs);
int[] result = new int[numClasses];
result[maxIndex] = 1;
return result;
}
``````

``````loop maxEpochs times
use weight gradients to update all weights
use bias gradients to update all biases
end-loop
``````

``````public void Train(double[][] trainData, int maxEpochs,
double learnRate, double decay)
{
double[] targets = new double[numClasses];
int msgInterval = maxEpochs / 10;
int epoch = 0;
while (epoch < maxEpochs)
{
++epoch;
...
``````

``````if (epoch % msgInterval == 0 && epoch != maxEpochs)
{
double mse = Error(trainData);
Console.Write("epoch = " + epoch);
Console.Write(" error = " + mse.ToString("F4"));
double acc = Accuracy(trainData);
Console.WriteLine(" accuracy = " + acc.ToString("F4"));
}
``````

``````double[][] weightGrads = MakeMatrix(numFeatures, numClasses);
``````

``````for (int j = 0; j < numClasses; ++j) {
for (int i = 0; i < numFeatures; ++i) {
for (int r = 0; r < trainData.Length; ++r) {
double[] outputs = ComputeOutputs(trainData[r]);
for (int k = 0; k < numClasses; ++k)
targets[k] = trainData[r][numFeatures + k];
double input = trainData[r][i];
weightGrads[i][j] += (targets[j] - outputs[j]) * input;
}
}
}
``````

``````double input = trainData[r][i];
int sign = (input > 0.0) ? 1 : -1;
weightGrads[i][j] += (targets[j] - outputs[j]) * sign;
``````

``````for (int j = 0; j < numClasses; ++j) {
for (int i = 0; i < numFeatures; ++i) {
for (int r = 0; r < trainData.Length; ++r) {
double[] outputs = ComputeOutputs(trainData[r]);
for (int k = 0; k < numClasses; ++k)
targets[k] = trainData[r][numFeatures + k];
double input = 1; // 1 is a dummy input
biasGrads[j] += (targets[j] - outputs[j]) * input;
}
}
}
``````

``````for (int i = 0; i < numFeatures; ++i) {
for (int j = 0; j < numClasses; ++j) {
weights[i][j] *= (1 - decay);  // wt decay
}
}
``````

``````...
for (int j = 0; j < numClasses; ++j) {
biases[j] *= (1 - decay);
}
} // While
} // Train
``````

## 總結

James McCaffrey為在華盛頓州雷蒙德市的微軟研究院工作和曾在幾個 Microsoft 產品包括 Internet Explorer 和 Bing。  博士。 麥卡弗裡也可以撥打 jammc@microsoft.com