(1)f(x)=√3/2sin2x-1/2cos2x=sin(2x-π/6)
x∈[5π/24,3π/4],∴2x-π/6∈[π/4,4π/3]
∴f(x)∈[-√3/2,1]
∴f(x)最小值为-√3/2,此时2x-π/6=4π/3,x=3π/4
f(x)最大值1,此时2x-π/6=π/2,x=π/3
(2)f(C)=0
∴2C-π/6=0、π
C=π/12、7π/12
∵sinB=2sinA
∴b=2a
∵c²=a²+b²-2abcosC
∴3=a²+4a²-4a²·(√6+√2)/4或者3=a²+4a²+4a²·(√6-√2)/4
∴