Woodfuels are the most heavily used energy source in sub-Saharan Africa. We analyzed the ecological impacts and modes of access of five user groups (domestic consumers, gin distillers, brick manufacturers, charcoal producers, and tea companies) drawing biomass energy from natural forests in western Uganda. While domestic consumers use the most species for fuelwood (>50), their consumption is likely sustainable because they generally harvest fast-growing species from fallows on their own land or their neighbors’. Charcoal producers prefer old-growth hardwood species and are responsible for the greatest loss of natural forests. They access forests by finding landholders who, either willingly or through coercion, allow trees on their lands to be cleared. The impact of charcoal production is exacerbated by a license system that undervalues natural forests and rewards rapid harvests across large areas. The tea industry consumes mainly eucalyptus wood (Eucalyptus spp.) from corporate plantations, but they indirectly create pressure on natural forests by hiring immigrants who subsequently settle in and clear forest remnants. If such practices continue, forest remnants will soon be exhausted, leaving Kibale National Park as the last natural forest in the region. Forest remnants are a vital source of water, medicinal plants, and energy for local citizens and to protect them from over-exploitation, policy makers should target the charcoal and tea industry for reform. Support for local land management institutions governing access to fallows and successional forests will inevitably enhance the policy interventions.