rTop-k: A Statistical Estimation Approach to Distributed SGD

Submitted by admin on Tue, 06/11/2024 - 01:30
The large communication cost for exchanging gradients between different nodes significantly limits the scalability of distributed training for large-scale learning models. Motivated by this observation, there has been significant recent interest in techniques that reduce the communication cost of distributed Stochastic Gradient Descent (SGD), with gradient sparsification techniques such as top-k and random-k shown to be particularly effective.

Author Names in Native Languages

Submitted by admin on Tue, 06/11/2024 - 01:28

The January 2023 issue of IEEE JSAC is a special issue on “Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications”. The phrase “semantic communications” started to conquer a significant real estate in the overall discussion on future wireless systems; yet, it sometimes remains fuzzy what the objective and scope of it is. The tutorial article written by the Guest Editors is an excellently written piece that brings clarity to the discourse on semantic communications.

INSIGHTS

Submitted by admin on Tue, 06/11/2024 - 01:19

 The January 2023 issue of IEEE JSAC is a special issue on “Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications”. The phrase “semantic communications” started to conquer a significant real estate in the overall discussion on future wireless systems; yet, it sometimes remains fuzzy what the objective and scope of it is. The tutorial article written by the Guest Editors is an excellently written piece that brings clarity to the discourse on semantic communications.

Propose a Special Issue

The IEEE Journal on Selected Areas in Information Theory (JSAIT) seeks high quality technical papers on all aspects of Information Theory and its applications. JSAIT is a multi-disciplinary journal of special issues focusing on the intersections of information theory with fields such as machine learning, statistics, genomics, neuroscience, theoretical computer science, and physics.

Information for Authors

The IEEE Journal on Special Areas in Information Theory  JSAIT is a multi-disciplinary journal of special issues focusing on the intersections of information theory with fields such as machine learning, statistics, genomics, neuroscience, theoretical computer science, and physics. Any field that utilizes the fundamentals of information theory, including concepts such as entropy, compression, coding, mutual information, divergence, capacity, and rate distortion theory is a candidate for a JSAIT special issue.