用神经网络分类上和下
( A, B )---3*30*2---( 1, 0 )( 0, 1 )
做一个网络，输入为3个点，训练集A,B各有4张图片。让B的4张图片全是0.排列组合A，记录迭代次数平均值的变化。收敛误差为7e-4，每个网络收敛199次。
其中得到一组数据
| 差值结构 | 1-A-B | 迭代次数 | 差值结构 | 2-A-B | 迭代次数 | |||||
| 0 | 0 | 0 | 0*3*1*6-0*0*0*0 | 6394.4774 | 1 | 1 | 0 | 6*1*3*0-0*0*0*0 | 7134.0452 | |
| 0 | 1 | 1 | 0*3*1*6-0*0*0*0 | 6394.4774 | 0 | 0 | 1 | 6*1*3*0-0*0*0*0 | 7134.0452 | |
| 0 | 0 | 1 | 0*3*1*6-0*0*0*0 | 6394.4774 | 0 | 1 | 1 | 6*1*3*0-0*0*0*0 | 7134.0452 | |
| 1 | 1 | 0 | 0*3*1*6-0*0*0*0 | 6394.4774 | 0 | 0 | 0 | 6*1*3*0-0*0*0*0 | 7134.0452 | |
| 0 | 0 | 0 | 0*6*4*3-0*0*0*0 | 6408.3869 | 0 | 1 | 1 | 3*4*6*0-0*0*0*0 | 7174.8894 | |
| 1 | 1 | 0 | 0*6*4*3-0*0*0*0 | 6408.3869 | 1 | 0 | 0 | 3*4*6*0-0*0*0*0 | 7174.8894 | |
| 1 | 0 | 0 | 0*6*4*3-0*0*0*0 | 6408.3869 | 1 | 1 | 0 | 3*4*6*0-0*0*0*0 | 7174.8894 | |
| 0 | 1 | 1 | 0*6*4*3-0*0*0*0 | 6408.3869 | 0 | 0 | 0 | 3*4*6*0-0*0*0*0 | 7174.8894 | |
| 0 | 1 | 1 | 3*1*6*0-0*0*0*0 | 6488.0352 | 0 | 0 | 0 | 0*6*1*3-0*0*0*0 | 7141.7538 | |
| 0 | 0 | 1 | 3*1*6*0-0*0*0*0 | 6488.0352 | 1 | 1 | 0 | 0*6*1*3-0*0*0*0 | 7141.7538 | |
| 1 | 1 | 0 | 3*1*6*0-0*0*0*0 | 6488.0352 | 0 | 0 | 1 | 0*6*1*3-0*0*0*0 | 7141.7538 | |
| 0 | 0 | 0 | 3*1*6*0-0*0*0*0 | 6488.0352 | 0 | 1 | 1 | 0*6*1*3-0*0*0*0 | 7141.7538 | |
| 1 | 1 | 0 | 6*4*3*0-0*0*0*0 | 6390.9497 | 0 | 0 | 0 | 0*3*4*6-0*0*0*0 | 7112.809 | |
| 1 | 0 | 0 | 6*4*3*0-0*0*0*0 | 6390.9497 | 0 | 1 | 1 | 0*3*4*6-0*0*0*0 | 7112.809 | |
| 0 | 1 | 1 | 6*4*3*0-0*0*0*0 | 6390.9497 | 1 | 0 | 0 | 0*3*4*6-0*0*0*0 | 7112.809 | |
| 0 | 0 | 0 | 6*4*3*0-0*0*0*0 | 6390.9497 | 1 | 1 | 0 | 0*3*4*6-0*0*0*0 | 7112.809 | |
| 0 | 0 | 1 | 1*6*0*3-0*0*0*0 | 6462.3166 | 0 | 1 | 1 | 3*0*6*1-0*0*0*0 | 7140.9397 | |
| 1 | 1 | 0 | 1*6*0*3-0*0*0*0 | 6462.3166 | 0 | 0 | 0 | 3*0*6*1-0*0*0*0 | 7140.9397 | |
| 0 | 0 | 0 | 1*6*0*3-0*0*0*0 | 6462.3166 | 1 | 1 | 0 | 3*0*6*1-0*0*0*0 | 7140.9397 | |
| 0 | 1 | 1 | 1*6*0*3-0*0*0*0 | 6462.3166 | 0 | 0 | 1 | 3*0*6*1-0*0*0*0 | 7140.9397 | |
| 1 | 0 | 0 | 4*3*0*6-0*0*0*0 | 6486.1156 | 1 | 1 | 0 | 6*0*3*4-0*0*0*0 | 7181.9447 | |
| 0 | 1 | 1 | 4*3*0*6-0*0*0*0 | 6486.1156 | 0 | 0 | 0 | 6*0*3*4-0*0*0*0 | 7181.9447 | |
| 0 | 0 | 0 | 4*3*0*6-0*0*0*0 | 6486.1156 | 0 | 1 | 1 | 6*0*3*4-0*0*0*0 | 7181.9447 | |
| 1 | 1 | 0 | 4*3*0*6-0*0*0*0 | 6486.1156 | 1 | 0 | 0 | 6*0*3*4-0*0*0*0 | 7181.9447 | |
| 1 | 1 | 0 | 6*0*3*1-0*0*0*0 | 6379.392 | 0 | 0 | 1 | 1*3*0*6-0*0*0*0 | 7126.4573 | |
| 0 | 0 | 0 | 6*0*3*1-0*0*0*0 | 6379.392 | 0 | 1 | 1 | 1*3*0*6-0*0*0*0 | 7126.4573 | |
| 0 | 1 | 1 | 6*0*3*1-0*0*0*0 | 6379.392 | 0 | 0 | 0 | 1*3*0*6-0*0*0*0 | 7126.4573 | |
| 0 | 0 | 1 | 6*0*3*1-0*0*0*0 | 6379.392 | 1 | 1 | 0 | 1*3*0*6-0*0*0*0 | 7126.4573 | |
| 0 | 1 | 1 | 3*0*6*4-0*0*0*0 | 6444.2764 | 1 | 0 | 0 | 4*6*0*3-0*0*0*0 | 7173.6533 | |
| 0 | 0 | 0 | 3*0*6*4-0*0*0*0 | 6444.2764 | 1 | 1 | 0 | 4*6*0*3-0*0*0*0 | 7173.6533 | |
| 1 | 1 | 0 | 3*0*6*4-0*0*0*0 | 6444.2764 | 0 | 0 | 0 | 4*6*0*3-0*0*0*0 | 7173.6533 | |
| 1 | 0 | 0 | 3*0*6*4-0*0*0*0 | 6444.2764 | 0 | 1 | 1 | 4*6*0*3-0*0*0*0 | 7173.6533 | |
这16组数据，左侧为第1列，右侧的为第2列。第1列的迭代次数全都小于第2列， 并且第1列和第2列的差值结构都是上下对称的，比如前3组
| 1 | 2 | |||||
| 0 | 0 | 0 | 1 | 1 | 0 | |
| 0 | 1 | 1 | 0 | 0 | 1 | |
| 0 | 0 | 1 | 0 | 1 | 1 | |
| 1 | 1 | 0 | 0 | 0 | 0 | |
| 0 | 0 | 0 | 0 | 1 | 1 | |
| 1 | 1 | 0 | 1 | 0 | 0 | |
| 1 | 0 | 0 | 1 | 1 | 0 | |
| 0 | 1 | 1 | 0 | 0 | 0 | |
| 0 | 1 | 1 | 0 | 0 | 0 | |
| 0 | 0 | 1 | 1 | 1 | 0 | |
| 1 | 1 | 0 | 0 | 0 | 1 | |
| 0 | 0 | 0 | 0 | 1 | 1 |
左右两侧的结构是对称的，但迭代次数确不相同，这种对称性被破缺了，神经网络到底是如何判断哪个是上，哪个是下的？
比较二者的结构
| 0*3*1*6-0*0*0*0 | ||||||||||
| 0 | 0 | 0 | → | 0 | 1 | 1 | → | 1 | 1 | 0 |
| 0 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | ||
| 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | ||
| 1 | 1 | 0 | ||||||||
如果略去全是0的一行，第1列的结构都可以变换成上三角矩阵。
| 3*4*6*0-0*0*0*0 | ||||||||||||||
| 0 | 1 | 1 | → | 0 | 1 | 1 | → | 1 | 1 | 0 | → | 1 | 0 | 0 |
| 1 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | |||
| 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | |||
| 0 | 0 | 0 |
而第2列的结构经变换后得到的都是下三角矩阵。
所以上三角矩阵的迭代次数是小于下三角矩阵的迭代次数的，
| A | B | |||||
| 1 | 1 | 0 | 〈 | 0 | 0 | 0 |
| 0 | 1 | 1 | 1 | 0 | 0 | |
| 0 | 0 | 1 | 1 | 1 | 0 | |
| 0 | 0 | 0 | 0 | 1 | 1 |
质心越低的迭代次数越大。所以对于这种特别的情况,用神经网络分类上下是可能的。尽管差值结构可以按照行1→2→3→4→1的顺序随意的变换而不改变迭代次数，但这种变换本身并不会改变形态内在的上下特征，这意味这神经网络各行的权重是不同的。而差值结构的列都可以按照1→2→3→1的顺序变换而不改变迭代次数，如
| 0 | 0 | 0 | 0*3*1*6-0*0*0*0 | 6394.477387 |
| 0 | 1 | 1 | 0*3*1*6-0*0*0*0 | 6394.477387 |
| 0 | 0 | 1 | 0*3*1*6-0*0*0*0 | 6394.477387 |
| 1 | 1 | 0 | 0*3*1*6-0*0*0*0 | 6394.477387 |
| 0 | 0 | 0 | 0*6*2*5-0*0*0*0 | 6478.336683 |
| 1 | 1 | 0 | 0*6*2*5-0*0*0*0 | 6478.336683 |
| 0 | 1 | 0 | 0*6*2*5-0*0*0*0 | 6478.336683 |
| 1 | 0 | 1 | 0*6*2*5-0*0*0*0 | 6478.336683 |
| 0 | 0 | 0 | 0*5*4*3-0*0*0*0 | 6415.944724 |
| 1 | 0 | 1 | 0*5*4*3-0*0*0*0 | 6415.944724 |
| 1 | 0 | 0 | 0*5*4*3-0*0*0*0 | 6415.944724 |
| 0 | 1 | 1 | 0*5*4*3-0*0*0*0 | 6415.944724 |
这说明神经网络各列是同权的，无差别，所以如果神经网络有质心，应该是到边的而不是到中心的。
| 差值结构 | 1-A-B | 迭代次数 | 差值结构 | 2-A-B | 迭代次数 | ||||||||||||||
| 1 | 1 | 0 | 0 | 0 | 0 | 0*3*1*6-0*0*0*0 | 6394.4774 | 1 | 1 | 0 | 6*1*3*0-0*0*0*0 | 7134.0452 | 0 | 0 | 1 | ||||
| 0 | 1 | 1 | 0 | 1 | 1 | 0*3*1*6-0*0*0*0 | 6394.4774 | 0 | 0 | 1 | 6*1*3*0-0*0*0*0 | 7134.0452 | 0 | 1 | 1 | ||||
| 0 | 0 | 1 | 0 | 0 | 1 | 0*3*1*6-0*0*0*0 | 6394.4774 | 0 | 1 | 1 | 6*1*3*0-0*0*0*0 | 7134.0452 | 1 | 1 | 0 | ||||
| 1 | 1 | 0 | 0*3*1*6-0*0*0*0 | 6394.4774 | 0 | 0 | 0 | 6*1*3*0-0*0*0*0 | 7134.0452 | ||||||||||
| 0 | 1 | 1 | 0 | 0 | 0 | 0*6*4*3-0*0*0*0 | 6408.3869 | 0 | 1 | 1 | 3*4*6*0-0*0*0*0 | 7174.8894 | 1 | 0 | 0 | ||||
| 1 | 1 | 0 | 1 | 1 | 0 | 0*6*4*3-0*0*0*0 | 6408.3869 | 1 | 0 | 0 | 3*4*6*0-0*0*0*0 | 7174.8894 | 1 | 1 | 0 | ||||
| 1 | 0 | 0 | 1 | 0 | 0 | 0*6*4*3-0*0*0*0 | 6408.3869 | 1 | 1 | 0 | 3*4*6*0-0*0*0*0 | 7174.8894 | 0 | 1 | 1 | ||||
| 0 | 1 | 1 | 0*6*4*3-0*0*0*0 | 6408.3869 | 0 | 0 | 0 | 3*4*6*0-0*0*0*0 | 7174.8894 | ||||||||||
| 1 | 1 | 0 | 0 | 1 | 1 | 3*1*6*0-0*0*0*0 | 6488.0352 | 0 | 0 | 0 | 0*6*1*3-0*0*0*0 | 7141.7538 | 0 | 0 | 1 | ||||
| 0 | 1 | 1 | 0 | 0 | 1 | 3*1*6*0-0*0*0*0 | 6488.0352 | 1 | 1 | 0 | 0*6*1*3-0*0*0*0 | 7141.7538 | 0 | 1 | 1 | ||||
| 0 | 0 | 1 | 1 | 1 | 0 | 3*1*6*0-0*0*0*0 | 6488.0352 | 0 | 0 | 1 | 0*6*1*3-0*0*0*0 | 7141.7538 | 1 | 1 | 0 | ||||
| 0 | 0 | 0 | 3*1*6*0-0*0*0*0 | 6488.0352 | 0 | 1 | 1 | 0*6*1*3-0*0*0*0 | 7141.7538 | ||||||||||
| 0 | 1 | 1 | 1 | 1 | 0 | 6*4*3*0-0*0*0*0 | 6390.9497 | 0 | 0 | 0 | 0*3*4*6-0*0*0*0 | 7112.809 | 1 | 0 | 0 | ||||
| 1 | 1 | 0 | 1 | 0 | 0 | 6*4*3*0-0*0*0*0 | 6390.9497 | 0 | 1 | 1 | 0*3*4*6-0*0*0*0 | 7112.809 | 1 | 1 | 0 | ||||
| 1 | 0 | 0 | 0 | 1 | 1 | 6*4*3*0-0*0*0*0 | 6390.9497 | 1 | 0 | 0 | 0*3*4*6-0*0*0*0 | 7112.809 | 0 | 1 | 1 | ||||
| 0 | 0 | 0 | 6*4*3*0-0*0*0*0 | 6390.9497 | 1 | 1 | 0 | 0*3*4*6-0*0*0*0 | 7112.809 | ||||||||||
| 1 | 1 | 0 | 0 | 0 | 1 | 1*6*0*3-0*0*0*0 | 6462.3166 | 0 | 1 | 1 | 3*0*6*1-0*0*0*0 | 7140.9397 | 0 | 0 | 1 | ||||
| 0 | 1 | 1 | 1 | 1 | 0 | 1*6*0*3-0*0*0*0 | 6462.3166 | 0 | 0 | 0 | 3*0*6*1-0*0*0*0 | 7140.9397 | 0 | 1 | 1 | ||||
| 0 | 0 | 1 | 0 | 0 | 0 | 1*6*0*3-0*0*0*0 | 6462.3166 | 1 | 1 | 0 | 3*0*6*1-0*0*0*0 | 7140.9397 | 1 | 1 | 0 | ||||
| 0 | 1 | 1 | 1*6*0*3-0*0*0*0 | 6462.3166 | 0 | 0 | 1 | 3*0*6*1-0*0*0*0 | 7140.9397 | ||||||||||
| 0 | 1 | 1 | 1 | 0 | 0 | 4*3*0*6-0*0*0*0 | 6486.1156 | 1 | 1 | 0 | 6*0*3*4-0*0*0*0 | 7181.9447 | 1 | 0 | 0 | ||||
| 1 | 1 | 0 | 0 | 1 | 1 | 4*3*0*6-0*0*0*0 | 6486.1156 | 0 | 0 | 0 | 6*0*3*4-0*0*0*0 | 7181.9447 | 1 | 1 | 0 | ||||
| 1 | 0 | 0 | 0 | 0 | 0 | 4*3*0*6-0*0*0*0 | 6486.1156 | 0 | 1 | 1 | 6*0*3*4-0*0*0*0 | 7181.9447 | 0 | 1 | 1 | ||||
| 1 | 1 | 0 | 4*3*0*6-0*0*0*0 | 6486.1156 | 1 | 0 | 0 | 6*0*3*4-0*0*0*0 | 7181.9447 | ||||||||||
| 1 | 1 | 0 | 1 | 1 | 0 | 6*0*3*1-0*0*0*0 | 6379.392 | 0 | 0 | 1 | 1*3*0*6-0*0*0*0 | 7126.4573 | 0 | 0 | 1 | ||||
| 0 | 1 | 1 | 0 | 0 | 0 | 6*0*3*1-0*0*0*0 | 6379.392 | 0 | 1 | 1 | 1*3*0*6-0*0*0*0 | 7126.4573 | 0 | 1 | 1 | ||||
| 0 | 0 | 1 | 0 | 1 | 1 | 6*0*3*1-0*0*0*0 | 6379.392 | 0 | 0 | 0 | 1*3*0*6-0*0*0*0 | 7126.4573 | 1 | 1 | 0 | ||||
| 0 | 0 | 1 | 6*0*3*1-0*0*0*0 | 6379.392 | 1 | 1 | 0 | 1*3*0*6-0*0*0*0 | 7126.4573 | ||||||||||
| 0 | 1 | 1 | 0 | 1 | 1 | 3*0*6*4-0*0*0*0 | 6444.2764 | 1 | 0 | 0 | 4*6*0*3-0*0*0*0 | 7173.6533 | 1 | 0 | 0 | ||||
| 1 | 1 | 0 | 0 | 0 | 0 | 3*0*6*4-0*0*0*0 | 6444.2764 | 1 | 1 | 0 | 4*6*0*3-0*0*0*0 | 7173.6533 | 1 | 1 | 0 | ||||
| 1 | 0 | 0 | 1 | 1 | 0 | 3*0*6*4-0*0*0*0 | 6444.2764 | 0 | 0 | 0 | 4*6*0*3-0*0*0*0 | 7173.6533 | 0 | 1 | 1 | ||||
| 1 | 0 | 0 | 3*0*6*4-0*0*0*0 | 6444.2764 | 0 | 1 | 1 | 4*6*0*3-0*0*0*0 | 7173.6533 | ||||||||||
下一篇：从0开始学python -26