Can anyone other than Assignee view or edit the Tasks?

Everyone can view Task but if the User is from the same office, they can only edit the Subject and Description of the Task. Whereas an Assignee can edit entire task details.