An Information-Theoretic Approach to Unsupervised Feature Selection for High-Dimensional Data

Submitted by admin on Tue, 06/11/2024 - 01:30

In this paper, we propose an information-theoretic approach to design the functional representations to extract the hidden common structure shared by a set of random variables. The main idea is to measure the common information between the random variables by Watanabe's total correlation, and then find the hidden attributes of these random variables such that the common information is reduced the most given these attributes.

Information-Theoretic Lower Bounds for Compressive Sensing With Generative Models

Submitted by admin on Tue, 06/11/2024 - 01:30
It has recently been shown that for compressive sensing, significantly fewer measurements may be required if the sparsity assumption is replaced by the assumption the unknown vector lies near the range of a suitably-chosen generative model. In particular, in (Bora et al., 2017) it was shown roughly O(k log L) random Gaussian measurements suffice for accurate recovery when the generative model is an L-Lipschitz function with bounded k-dimensional inputs, and O(kdlogw) measurements suffice when the generative model is a k-input ReLU network with depth d and width w.

Secure Source Coding Resilient Against Compromised Users via an Access Structure

Submitted by admin on Tue, 06/11/2024 - 01:30
Consider a source and multiple users who observe the iid copies of correlated Gaussian random variables. The source wishes to compress its observations and store the result in a public database such that (i) authorized sets of users are able to reconstruct the source with a certain distortion level, and (ii) information leakage to non-authorized sets of colluding users is minimized. In other words, the recovery of the source is restricted to a predefined access structure.

Information-Theoretic Tools to Understand Distributed Source Coding in Neuroscience

Submitted by admin on Tue, 06/11/2024 - 01:30
This paper brings together topics of two of Berger’s main contributions to information theory: distributed source coding, and living information theory. Our goal is to understand which information theory techniques can be helpful in understanding a distributed source coding strategy used by the natural world. Towards this goal, we study the example of the encoding of location of an animal by grid cells in its brain.

On the Fundamental Limit of Distributed Learning With Interchangable Constrained Statistics

Submitted by admin on Tue, 06/11/2024 - 01:30
In the popular federated learning scenarios, distributed nodes often represent and exchange information through functions or statistics of data, with communicative processes constrained by the dimensionality of transmitted information. This paper investigates the fundamental limits of distributed parameter estimation and model training problems under such constraints. Specifically, we assume that each node can observe a sequence of i.i.d. sampled data and communicate statistics of the observed data with dimensionality constraints.

Summary Statistic Privacy in Data Sharing

Submitted by admin on Tue, 06/11/2024 - 01:30
We study a setting where a data holder wishes to share data with a receiver, without revealing certain summary statistics of the data distribution (e.g., mean, standard deviation). It achieves this by passing the data through a randomization mechanism. We propose summary statistic privacy, a metric for quantifying the privacy risk of such a mechanism based on the worst-case probability of an adversary guessing the distributional secret within some threshold.

Straggler-Resilient Differentially-Private Decentralized Learning

Submitted by admin on Tue, 06/11/2024 - 01:30
We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latencycomprising both computation and communication latency.

Robust Algorithmic Recourse Under Model Multiplicity With Probabilistic Guarantees

Submitted by admin on Tue, 06/11/2024 - 01:30
There is an emerging interest in generating robust algorithmic recourse that would remain valid if the model is updated or changed even slightly. Towards finding robust algorithmic recourse (or counterfactual explanations), existing literature often assumes that the original model m and the new model M are bounded in the parameter space, i.e., ||Params(M)-Params(m)||<.>