Many African drylands are undergoing rapid change as open country is replaced by monocultures of Prosopis trees, a species that was brought to Africa in the 1980s from the Americas. While some local stakeholder groups have called for the complete removal of these trees others seek to promote them further, leading to conflicts over how best to manage the tree. To understand why one tree elicits such different responses, we used a novel approach, that combines data on how the tree impacts the human benefits provided by these dryland ecosystems with interviews of local inhabitants in which we assessed how much they prioritized each of these benefits. This showed that the impact of Prosopis trees changed depending on which benefits were considered most important. For example, charcoal producers highly value wood, and profit from Prosopis, whereas conservationists, which put high priority on biodiversity and animals related to ecotourism, were negatively affected. We argue that identifying these different viewpoints, and their implications, helps researchers and policy makers to understand the origin of land use conflicts like this and to ultimately develop land management systems in which all stakeholders receive the benefits they desire. For example, in the case of Prosopis, our results show a need for different land cover types, within a region, and the need to actively control Prosopis, so that it only covers a part of the area.