It won't make much difference whether you agree or not.为什么不是differences

difference 表示"差异,差别",可数:
There are many differences between living in a city and living in the country.生活在城市和生活在乡村有许多不同之处.
difference 表示"差额,差数",不可数:
The difference between 5 and 11 is 6.十一和五的差是六.
Flowers make no/a lot of difference to a room. 一个房间里有没有鲜花没有/有很大差别.
It doesn't make much/any/the least difference to me what you do .你干什么对我都没有多大/任何/丝毫关系.
When you're learning to drive, having a good teacher makes all the difference.初学驾驶如有好的老师指导,差别很大.
difference 表示"不和,分歧",可数:
They've settled their differences and are friends again.他们已消除芥蒂,重归于好.
所以说，这里的difference是不可数名词。

你同意与否，关系不大

因为是赞成与不赞成之间的区别,就一处

difference 作为不同点的时候，是可数名词；但是如果作为影响或者关系来讲的话，就是不可数名词。这句话翻译为：你同不同意没什么关系。