My brother is giving me his daughter, do both parents have to consent to that?

Yes, both birthparents must agree to terminate their parental rights for a legal adoption. Even if your brother has full legal custody, the birthmother must terminate her rights. If not, she can reclaim the child at anytime.

I encourage you to speak with an family law attorney to ensure that all the paperwork is done correctly to ensure permanency for the child.